Through grad school and my visiting position at Kalamazoo, I was blissfully shielded from the idea of assessment. The veil was lifted when I came to Shimer, and I must confess that after our first meeting to discuss the issue, I was horrified. I remain suspicious of the “education reform” movement in general, but I have come to think that assessment — at least as it is conceived by Shimer’s accrediting body — might not be a bad thing. If I imagine that I’d never heard of Arne Duncan or charter schools and someone came up to me and suggested that my school might want to clarify its goals, gather information about how it’s doing in meeting those goals, and change its practices if it’s not working, I would find it hard to object. It bears a cost in terms of time and energy — but then so does plugging away at something that’s not working. Paying a reasonable number of hours up front in terms of data collection seems like a reasonable “hedge” against wasting countless hours on a pointless boondoggle.
One concern, particularly in liberal arts circles, surrounds the numerical aspect of assessment. I share the skepticism that numbers can really “capture” what we do, but we already use numerical forms of assessment all the time — we just mostly translate them into alphabetical form. It’s also widely acknowledged that professors as a whole have wavered in their dedication to rigorous grading, due in large part to the increasingly disproportionate real-world consequences grades can have on their students’ lives. The “grade inflation” trend reportedly began as professors were unwilling to condemn a student to die in Vietnam because their term paper was too short, and the financial consequences of grades likely play a similar role today. Hence it makes sense to come up with a parallel “internal” system of measurement so that we can be more objective (and also measure progress on a single scale, without the difficulty of deciding whether a B in an advanced seminar denotes an advance over an A in a freshman intro class, etc.).
When it comes to cross-institutional comparisons, however, we use a much more insidious numerical measure to signal educational quality: price. Education is a perverse market where everyone is in competition to charge more, because that’s how you signal that you’re an elite institution. There are other, more detailed measures such as those collected by the widely derided U.S. News and World Report ranking system — but those measures largely treat the educational process itself as a black box, and they’re incredibly easy to game. (For instance, I recently learned of an institution that decided to replace all its contingent faculty with full-time, non-tenure-track faculty at one fell swoop, due solely to a change in the U.S. News ranking algorithm.)
Obviously the problem with the assessment movement is that it’s embedded in a neoliberal logic whereby the government’s role is to make sure markets function as much as possible. Ideally, of course, there would be a robust “public option” in the form of well-funded state schools that provide a quality education for minimal cost, which would automatically bring down prices across the board. Unfortunately, we do not live in a world where that option is seriously entertained — everything has to be achieved via a “bank shot.” In this case, they’re trying to create a market based on demonstrable effectiveness rather than who can charge the most and/or offer the most extra-curricular amenities. The reasoning is presumably that once consumers have this more robust and relevant information, they won’t tolerate inflated tuition for a school that delivers the same basic outcome as a community college (for example). The government is also likely going to play a more forceful role by increasing the conditionality of financial aid, which would mean that one of the biggest drivers of tuition inflation, namely the unlimited government subsidy of whatever the school feels like charging, would be under control. (I assume this is part of what’s going on with the consolidation of control of the student loan program more directly in the hands of the Department of Education.)
It will almost certainly turn out to be the case that this elaborate “bank shot” strategy will not deliver the promised results overall. It is also possible that in the future, the assessment regime will be harnessed to force greater homogeneity in the educational market. As it stands now, however, all indications are that Shimer’s accrediting body is shielding institutions from any homogenizing force and encouraging them to create assessment programs that reflect their own unique goals and institutional ethos. In other words, for now the question is not whether we’re measuring up to some arbitrary standard, but whether each institution can make the case that they are delivering on what they promise.
Hence — leaving aside for the moment the market-making effects assessment is supposed to have, as well as suspicions about how it could be used in the future — it seems possible to come up with an assessment system that would actually be helpful for figuring out how to be faithful to each school’s own goals. I have to admit that part of my sanguine attitude stems from the fact that Shimer’s pedagogy embodies what independent researchers have already demonstrated to be “best practices” in terms of discussion-centered, small classes — and so if we take the trouble to come up with a plausible way to measure what the program is doing for our students, I’m confident the results will be very strong. I’m also sure that there are some things that we’re doing that aren’t working as well as they could, but we have no way of really knowing that currently.
We all have limited energy and time, especially at a place like Shimer where the faculty needs to take on a lot of the burden of administration of the school in addition to teaching, and so anything that can help us to make sure we’re devoting our energy to things that are actually beneficial seems all to the good. At the same time, though, I recognize that not every school has the strong shared governance that Shimer does, and so the assessment regime will most likely be more arbitrary and unhelpful when designed by the autonomous administrative class that wants to “run the college like a business.” That’s a separate question from assessment as such, however — and I’d even suggest that faculty resistence to the very idea of assessment may further weaken faculty governance and allow the “bad” administrative types even more of a free hand.
So there you have it. I look forward to hearing about what a horrifying neoliberal sell-out I am.