Smelly Knowledge

It’s no secret that I’m not a big fan of the No Child Left Behind Act on a number of levels. Much of the time, the high level of accountability coupled with a lack of adequate funding and the idea of the Federal government intruding into the state enterprise of education are the most reported. Of course, the high levels of accountability — most often expressed in the form of high-stakes tests — have many (unforseen?) implications, such as higher levels of in-grade retention and a greater number of students dropping out of school. There are other reasons, from a practitioner-implementation perspective; Reality Testing NCLB and No Child Left Behind: An analysis of the controversy both give good accounts of some of these issues.

The implications on educational research, however, is often not discussed in public discourse — unless, of course, you are an educational researcher (see any volume of Educational Researcher published since NCLB appeared on the horizon and you’ll see what I mean). Part of the problem is that it is difficult to talk about educational research; it is not a concept that has entered the public imagination. It may be that the first reason for this is that when one thinks “researcher,” it’s likely that the first image that comes to mind is the mad scientist replete with white coat. If you add in long days and nights in the lab, minutely manipulating different variables, observing and making notes, and collecting and compiling data, you’ve got a pretty full picture of the public conception of the researcher and research (which is actually pretty close to my days in college genetics futzing around with fruit flies). Educational research just doesn’t seem to fit into this model, or does it?

The US Department of Education, as part of the NCLB suite of new requirements, sets out the following guidelines:

No Child Left Behind sets forth rigorous requirements to ensure that research is scientifically based. It moves the testing of educational practices toward the medical model used by scientists to assess the effectiveness of medications, therapies and the like. Studies that test random samples of the population and that involve a control group are scientifically controlled.

Moving “educational practices toward the medical model,” eh? The allopathic medical model is built in the assumption that a “sick” person — a patient or subject — lacks health. All medicines, drugs, and treatments are designed to restore whatever that notion of health is; it is a classic deficit model with no — until very recently — clear picture of maintaining wellness rather than combatting disease. This is one of the many reason so many people are spending their money on “alternative” or “complementary” medicine.

Is this what we want for educational research? Is poor performance on high-stakes tests really a lack of knowledge or a lack of understanding? Isn’t the whole thing much more complex? I think so. “Random sampling” and “control groups” are extremely difficult to isolate in a setting in which learning takes place. Not only that, but isn’t one of the things (well, at the very least, progressive) educators are looking for is the outcomes that are not and cannot be predicted?

A way of thinking about educational research came to me from the book Collapse: How Societies Choose to Fail or Succeed by geographer and biologist Jared Diamond. Dr. Diamond discusses the nature of science and describes his research methodology on pages 17-18:

Science is often misrepresented as “the body of knowledge acquired by performing replicated controlled experiments in the laboratory.” Actually, science is something much broader: the acquisition of reliable knowledge about the world.
…
When I began studying birds in New Guinea rainforest in 1964, I was immediately confronted with the problem of acquiring reliable knowledge without being able to resort to replicated controlled experiments, whether in the laboratory or outdoors. It’s usually neither feasible, legal, nor ethical to gain knowledge about birds by experimentally exterminating or manipulating their populations at one site while maintaining their populations at another site as unmanipulated controls.
…
A frequent solution is to apply what is termed the “comparative method” or the “natural experiment” — i.e., to compare natural situations differing with respect to the variable of interest.
…
In the present book… I compare many past and present societies that differed with respect to environmental fragility, relations with neighbors, political institutions, and other “input” variables postulated to influence a society’s stability. The “output” variables that I examine are collapse or survival, and form of the collapse if a collapse does occur. By relating output variables to input variables, I aim to tease out the influence of possible input variables on collapses.

This research model seems to me to fit the state of the educational system as well as the process in which learning takes place. The comparative method seems to address the following points, which the strict “medical” model does not, while still satisfying a need for reliability:

• A recognition of the complex systems in which learning is situated; the idea in the comparative method is to “tease out,” rather than isolate, as in the model advocated by the Department of Education.
• The comparative model is not as fixated on the idea of deficit or gain as the medical model; there is an implicit understanding that learning is multivariate, multifaceted, and complex.
• An increased potential for collecting meaningful qualitative data; qualitative information, in the form of narratives and stories, provides a wealth of information not necessarily conveyed in the numbers and statistics of quantitative data.
• An entry into educational research — a first step into reflective practice — for teachers; the medical model, with its emphasis on artificial contexts, is not necessarily conducive for teachers to become involved.

While I personally favor a very narrative approach — qualitative rather than quantitative — to educational research, I do believe that reliability is an important factor to consider and more quantitative methods have merit in answering many research questions. It seems that the comparative method is much better suited for the complexities and realities — as well as purposes — of education and learning than the medical model. Yet the purse strings are held by firm believers in the medical model with very little room for discussion.

The refusal to engage in dialogue is the most disheartening piece of the story.