Answering the “Hard” Questions

(Or, "Not an Article About Don Imus")

April 13, 2007

Dr. Watson Scott Swail, President, Educational Policy Institute

CHICAGO, IL — Further travels with me this week involve a visit to the American Educational Research Association Annual Meeting in Chicago, Illinois. I’m here with 15,000 of my closest friends. I think I’ve shaken hands with half of them. AERA is one of those conferences that forces one into a strategic mode just to navigate: sessions at four hotels, over 80 concurrent sessions at each time period. Heck, the conference book is quite literally the size of most city telephone books. It becomes the bible of the week for attendees.

Many of the attendees come for the pure reason of getting a peer-reviewed paper presented--looks nice on the resume. That stated, many of the sessions are somewhat suspect: grad students getting their first play. But once in a while a session pops up that takes your interest. Enter Pascarella and Terenzini.

Ernie Pascarella and Pat Terenzini are two names in higher education that resonate almost among all others. I might add Vince Tinto to that list, Bill Bowen and Derek Bok, and perhaps a few others. But it’s a very short list.

For the uninitiated, “Pascarella and Terenzini” (as they are typically referred) are the authors of the seminal academic work “How College Affects Students,” first published in 1991 with a redux in 2005. In both volumes, the two researchers reviewed thousands of research papers in an attempt to tell a story about the impact of college on students—a field of study that is understandably broad, but nonetheless important.

So it was a distinct pleasure to hear these two gentlemen speak here in Chicago yesterday afternoon. In fairness, they were joined by John Smart of the University of Memphis and Barbara Townsend of the University of Missouri – Columbia, and while those two individuals did a nice job talking about the issues impacting students and higher education, this was the Ernie and Pat show, for all intents and purposes.

A major point of Pat’s was his sense that the study of higher education was moving dangerously away from theory to anecdotal research. He said that in their earlier work, there was a definitive focus on theoretical-based research on college students. And while he praised the NSSE (National Survey of Student Engagement) and similar “research” conducted in the last decade, he was worried that too much was focused on non-theoretical ventures, pushing us toward learning without the aid of well-thought theoretical processes.

I’m not sure if Pat is right on this point, but it did make me think about it. More interesting, even as I write this week’s blog, an article popped up on my screen from this morning’s USA TODAY called “Usefulness of Education Research Questioned.” Too funny. In that article, Greg Toppo writes about the new political use (or misuse; or complete ignorance) of research for political purposes, with the example of an Education Department study released last year which showed that public school students, on average, do better than public school students. Of course, the Department ignored that study and proposed $100 million for voucher programs to send public school students to private schools. Great example.

There are two problems here. First is what Pat calls this shift from the theoretical to the anecdotal. And I agree, that can be a dangerous thing for us who try and link the research and policy worlds. If our basic research is based on what we “think” rather than what we “know,” we are surely in trouble. The second problem is the political use of research, as shown in the previous example. If the current administration, which has pushed evidence-based practice so far down the throats of researchers in the US, is willing to ignore its own high-level research in place of political expediency, then what is the point of any research, theoretical or anecdotal? This is a serious issue for us to content with, because if that latter holds true, then all our efforts must really be reconsidered.

As a researcher, I try to answer the difficult questions (as John Smart stated, we seem to be moving away from these in favor or the simple sound bite research) as well as possible. This is difficult not just because of the complexity of good, empirical study, but also of the business side of trying to find the funding to conduct these studies—no one wants to fund studies that might take 5-6 years to find answers.

But somehow, regardless of the 11,000 papers presented at this year’s AERA conference (no exaggeration), we must wonder how much of the research is worth the paper (or pdf file) that it is printed on (and how much toner is used in that process… I digress). Perhaps we need to retool our research efforts in education—not to shy away from ED’s focus on evidence-based study—to ensure that the research is usable in terms of molding educational practice in classrooms, lecture halls, and even residence halls. Perhaps (again) this is my call to philanthropic agencies and government agencies (including IES, but certainly not limited to) to allow for creative rather than prescriptive research. All of this research, by the way, would need to be underpinned by theoretical constructs, as Terenzini noted. But then at least we could produce research that, in turn, influenced the development of public policy which, in turn, influenced classroom practice. A novel thought.

So how much should ED spend on vouchers next year?

Have a great weekend.