Competing for Quality

April 6, 2007

Alex Usher, Vice-President, Educational Policy Institute

Educational quality is hard to measure. Lord knows, it's been tried. Governments all over the world have been wrestling with this problem for years.

The history of measuring quality at the post-secondary level is not a pretty one. Left to themselves, institutions stuck with a system of periodic program review, which was neither transparent, nor to any real degree empirical. It smacked of a cosy arrangement amongst academics to make sure that nothing ever changed.

In the 1980s and 1990s, governments declared themselves unsatisfied with this and started measuring performance (and tying funding) to a set of simple, easy-to-understand, numeric indicators, most of which were simple outcomes measures such as graduation rates and employment rates. Indeed, at one point, over 30 US states adopted some kind of performance-based funding.

This, needless to say, was as much of a disaster as the previous arrangement had been. Graduation rates can be manipulated by raising entry requirements or lowering academic standards. Institutional employment rates are largely a function of geography and the range of courses offered. Performance-based funding, in other words, did not incentivize any kind of behaviour that normal, rational people might think was actually beneficial.

There have also been other experiments in quality. In Europe and Australia, there has been a tendency to equate quality not with numerical outputs but with process - that is to say, that quality is defined as a process which leads to improvement. This is a step forward over performance indicators in a couple of ways. First, by focusing over process rather than input, it tends to democratize quality discussions within institutions, putting less authority in the hands of the number-crunchers and back into the classrooms and common rooms. Second, by emphasizing that quality needs to be a continual concern, something that *always* needs to be dealt with.

But this idea has its limits, too. The most important one being that process on its own doesn't necessarily lead to change; andmoreover, it's not always clear what kinds of behaviour or outcomes governments should be trying to incentivize.

Yet this kind of problem shouldn't be that hard to solve. What do we really want from institutions in terms of quality, anyway? More than anything, we want them to constantly thinkabout quality, we want them to make greater efforts to measure what it is they do and how successful they are at dong it, and we want them to make continual adjustments to improve their offerings.

Ok, then: then let's incentivize thinking about, measuring and improving quality.

Imagine a system where pots of money were available on a competitive basis to develop new methods of measuring quality. Imagine if every year, a competition were held to award a few million dollars to the institution(s) who came up with the best plan to improve teaching quality. Imagine if every year there was a $5 million prize for the institution that had improved the most in the commercialization of research, or internationalizing its curriculum. Even if an institution didn't win in a given year, it would still have spent a lot of time thinking about how to improve its offerings and some of that thinking would no doubt seep through to programs even in the absence of new money.

As for institutions that won: well, I think we'd likely see some dramatic improvements. If we made quality measurement a mandatory part of the competition, then we would even have solid empirical evidence about the effectiveness of the changes.

Seriously, how great would that be?

I'm willing to bet these prizes would generate a lot of new and creative thinking about these issues - and it would leverage a lot more systemic change than any performance funding envelope could. As quality becomes increasingly important, it's this kind of systemic change we need to be looking for.

Enjoy the weekend.