• from The Columbus Dispatch - "Much education research lacks proper standard"

Back to List of Articles

Much education research lacks proper standard

June 16, 2003

By William L. Bainbridge

A few years ago, late-night TV host Jay Leno found fodder for his opening monologue in a national study purporting to be educational research. A major finding was that students who took algebra in high school did better in college than those who did not. Leno pointed out the obvious: Students in the college preparatory track are required to take algebra, and the research money was spent proving that those who prepare for college do better in college than those who did not.

Such wheel-spinning research wastes taxpayer money and is not uncommon. And thereís nothing funny about that.

One example is research that purports to show improvements in student achievement as a result of new, presumably better curricular programs. More often than not, we find that when great improvements in test scores emerge, the teaching staff has been "reconstituted." In a nutshell, the principal has removed weak teachers or those who are not compatible with curricular changes. Because there is a great body of research indicating teachers are the key elements in the education process, when the majority of the teachers have changed, it is very difficult to attribute gains in student performance to any curriculum. The changes might simply be due to increasing the quality of the teaching. Better teachers may mean better student outcomes regardless of the curriculum. At any rate, attribution of results is suspect because the causes cannot be determined.

In similar studies, a little deeper probing reveals the student body changed through high dropout rates or transfers, thereby leaving higher-performing students to be tested. In other investigations, computers in homes have been linked with high scores, but most people know that students from high socioeconomic homes tend to score better than those from low socioeconomic homes. And, obviously, students in families in the higher income brackets are the ones better able to afford home computers.

Too much federal money has been expended on correlation studies that, in many cases, end up documenting common sense, as these examples and Jay Leno suggest. Thoughtful researchers have pointed out flaws in similar studies for years. Problems ranging from sampling bias, experimenter bias, statistical flaws and errors in randomization have created a paucity of useful benchmarks in the body of educational research.

A breath of fresh air was introduced recently by a prominent education official who can actually deal with bias, errors and faulty reasoning in federally funded educational research. Grover J. "Russ" Whitehurst, the new director of the Institute of Education Sciences at the U.S. Department of Education, has the courage to point out the flaws of past practice in his vast agency. His responsibilities include gathering and reporting information on our progress in education, funding research on practices that improve academic achievement and opportunities and evaluating the effectiveness of education programs.

Whitehurst is borrowing from medicine the concept of randomized trials to investigate claims about the effects of an educational intervention on outcomes. Itís the same approach to research that has been designed to prove that hormone replacement therapy, welfare programs and the DARE drug-abuse-prevention program may actually produce negative consequences.

This approach is an outgrowth of what many of us learned in high school in the introduction to experimental science; control groups and experimental groups are examined to see if interventions make a difference. These simple measures reduce the possibility that researchersí biases will contaminate findings. Much educational research does not go far enough to implement these kinds of controls.

Educational research must not continue to be a resource for comedians. Whitehurst has it right. It is high time we protect our precious dollars and provide dynamic assistance to teaching and learning through the use of randomized trials.

is Distinguished Research Professor at the University of Dayton and is President & Chief Executive Officer of SchoolMatch®, a Columbus based educational auditing, research, data firm.

Back to List of Articles | SchoolMatch.com | About Us