- from The Columbus Dispatch - "Much education research lacks proper standard"
June 16, 2003
By William L. Bainbridge
|A few years ago, late-night TV host Jay Leno found fodder
for his opening monologue in a national study purporting to be educational
research. A major finding was that students who took algebra in high
school did better in college than those who did not. Leno pointed out the
obvious: Students in the college preparatory track are required to take
algebra, and the research money was spent proving that those who prepare
for college do better in college than those who did not. |
wheel-spinning research wastes taxpayer money and is not uncommon. And
thereís nothing funny about that.
|One example is research that purports to show improvements in student
achievement as a result of new, presumably better curricular programs.
More often than not, we find that when great improvements in test scores
emerge, the teaching staff has been "reconstituted." In a nutshell, the
principal has removed weak teachers or those who are not compatible with
curricular changes. Because there is a great body of research indicating
teachers are the key elements in the education process, when the majority
of the teachers have changed, it is very difficult to attribute gains in
student performance to any curriculum. The changes might simply be due to
increasing the quality of the teaching. Better teachers may mean better
student outcomes regardless of the curriculum. At any rate, attribution of
results is suspect because the causes cannot be determined. |
similar studies, a little deeper probing reveals the student body changed
through high dropout rates or transfers, thereby leaving higher-performing
students to be tested. In other investigations, computers in homes have
been linked with high scores, but most people know that students from high
socioeconomic homes tend to score better than those from low socioeconomic
homes. And, obviously, students in families in the higher income brackets
are the ones better able to afford home computers.
federal money has been expended on correlation studies that, in many
cases, end up documenting common sense, as these examples and Jay Leno
suggest. Thoughtful researchers have pointed out flaws in similar studies
for years. Problems ranging from sampling bias, experimenter bias,
statistical flaws and errors in randomization have created a paucity of
useful benchmarks in the body of educational research.
A breath of
fresh air was introduced recently by a prominent education official who
can actually deal with bias, errors and faulty reasoning in federally
funded educational research. Grover J. "Russ" Whitehurst, the new director
of the Institute of Education Sciences at the U.S. Department of
Education, has the courage to point out the flaws of past practice in his
vast agency. His responsibilities include gathering and reporting
information on our progress in education, funding research on practices
that improve academic achievement and opportunities and evaluating the
effectiveness of education programs.
Whitehurst is borrowing from
medicine the concept of randomized trials to investigate claims about the
effects of an educational intervention on outcomes. Itís the same approach
to research that has been designed to prove that hormone replacement
therapy, welfare programs and the DARE drug-abuse-prevention program may
actually produce negative consequences.
This approach is an
outgrowth of what many of us learned in high school in the introduction to
experimental science; control groups and experimental groups are examined
to see if interventions make a difference. These simple measures reduce
the possibility that researchersí biases will contaminate findings. Much
educational research does not go far enough to implement these kinds of
Educational research must not continue to be a resource
for comedians. Whitehurst has it right. It is high time we protect our
precious dollars and provide dynamic assistance to teaching and learning
through the use of randomized trials.
is Distinguished Research Professor at the
University of Dayton and is President & Chief Executive Officer
of SchoolMatch®, a Columbus based educational auditing, research, data