"Educational Research Should Be Nothing to Laugh At" By William L. Bainbridge. EducationNews.org. June 9, 2003.

from EducationNews.org

The Internetís Leading Source of EducationNews

Monday, June 09, 2003

EducationNews/Commentaries & Reports

EducationNews.org Commentaries and Reports

Educational Research Should Be Nothing to Laugh At
Monday, June 09, 2003

By William L. Bainbridge, Ph.D.


Educational Research Should Be Nothing to Laugh At

By William L. Bainbridge, Ph.D.

A few years ago, late-night talk show host Jay Leno found fodder for a joke during his opening monologue in a national study purporting to be educational research. A major finding of the project was that students who take algebra in high school do better in college than those who have not had algebra. Obviously, Leno pointed out, all of the students in the college preparatory track are required to take algebra, and therefore large amounts of money were spent essentially proving that those who prepare for college do better in college than those who did not originally plan or prepare to go on to higher education.

Those of us concerned about both school improvement and balancing federal expenditures on education were taken aback, not because the joke was so funny, but because such examples of wheel-spinning research, wastes taxpayer money.

Some other clear examples relate to research that purports to show improvements in student achievement as a result of new, and presumably better, curricular programs. More often than not, we find that when great improvements in test scores emerge, the staff has been "reconstituted." In a nutshell, this simply means the principal has removed weak teachers or those who are not compatible with curricular changes. Since there is a great body of research indicating teachers are the key elements in the education process, when the majority of the teachers have changed, it is very difficult to attribute gains in student performance to any curriculum. The changes might simply be due to increasing the quality or caliber of the instructional staff. Simply put, better teachers may mean better student outcomes regardless of the curriculum. At any rate, attribution of results is suspect because the causes cannot be determined.

In similar studies, a little deeper probing reveals the student body has changed through high dropout rates or transfers, thereby leaving only the higher performing students to be tested. In other investigations, computers in homes have been linked with high scores, but most people know that students from high socioeconomic homes tend to score better than those from low socioeconomic homes. Obviously, students from homes in the higher income brackets are the ones more able to afford home computers.

The point is this: too much federal money has been expended on correlation studies which, in many cases, end up documenting "common sense," as these examples suggest. Thoughtful researchers have pointed out flaws in similar education studies for years. Problems ranging from sampling bias, experimenter bias, statistical flaws and errors in randomization have created a paucity of useful benchmarking tools in the body of educational research that can effectively work in K-12 achievement.

Recently, a breath of fresh air has been introduced into the process by a prominent education official who can actually deal with bias, errors and faulty reasoning in federally funded educational research. His name is Grover J. "Russ" Whitehurst, the new Director of the Institute of Education Sciences at the U.S. Department of Education. Whitehurst has the courage to point out the flaws of past practice in his vast agency. His responsibilities include the gathering and reporting of information on our nationís condition and progress in education, funding research on practices that improve academic achievement and opportunities and evaluating the effectiveness of education programs. He is expected to marshall the resources of the National Center for Education Statistics, the National Center for Education Research and the National Center for Education Evaluation and Regional Assistance to improve student performance through better educational effectiveness research.

Whitehurst says his research priorities are effective instruction in reading, math and science; closing achievement gaps; teacher quality; standards and assessment and education finance. He is borrowing from medicine the concept of randomized trials to investigate claims about the effects of an educational intervention on outcomes. Itís the same approach to research that has been designed to prove that hormone replacement therapy, welfare programs and the DARE drug prevention program may actually create unanticipated negative consequences.

Since the summer of 2000, SchoolMatch, under the direction of psychologist and researcher Frederic K. Lawrence, has been engaged by two major national publishers to conduct what scientists call "quasi-experimental" research in school systems in various states. The process is very simple: using our databases, we identify large sets of school districts that are demographically similar in terms of parent education level, parent income level, poverty rates and English language fluency. In other words, they have similar student populations. Those schools that have introduced a new educational intervention within the group and those that have not are then identified. The task is to see if the intervention itself is having a measurable effect on student performance.

This approach is really just an outgrowth of what many of us learned in high school in the introduction to experimental science. Control groups and experimental groups are examined to see if interventions make a difference. These simple measures reduce the possibility that researchers' individual biases will contaminate the findings. Much current educational research does not go far enough to implement these kinds of controls.

Educational research must not continue to be a source of comedy. Director Whitehurst has it right. It is high time we protect our precious dollars and provide dynamic assistance to teaching and learning through the use of randomized trials.

William L. Bainbridge, Ph.D., is Distinguished Research Professor at the University of Dayton and President and Chief Executive Officer of SchoolMatch, a Columbus-based educational auditing, research and data firm.