Research into K-12 mathematics education.

Studies such as the Third International Mathematics and Science Study (TIMSS) and the National Assessment of Educational Progress (NAEP) paint an unfavorable picture of United States K-12 mathematics education: TIMSS shows that the relative performance of U.S. students decreases as they progress through the educational system, and the NAEP long term trend shows only very modest progress since the early 1970's and shows no clear relation between this small progress and any educational policy -- for all we know it is just a weak correlate of general economic prosperity.

There is no shortage of talk about mathematics education reform and plenty of government support for reform efforts, with preference for programs that are based on the Standards of the National Council of Teachers of Mathematics and on constructivist pedagogy and that emphasize process and "understanding" over ability. Unfortunately, it is hard for a professional mathematician that takes an interest in the matter to avoid the conclusion that on the whole these reform efforts are only pushing mathematics education in the wrong direction.

A practicing scientist might think that reform efforts could, should, and probably would be guided by a respected body of research into what works and what does not, although within such a body of research there might still be significant differences in research focus, methodology and results. With that in mind I started looking for appropriate research, and this letter is a little report on my search. I'll say right away that the outcome has been entirely negative.

I have looked for K-12 mathematics education research that has the following characteristics.

• Large scale longitudinal study. Many pupils from many schools followed over multiple years. The primary unit of analysis is the pupil, not the classroom or the school. Performance data are obtained at least once a year using broad tests -- sufficiently broad to avoid teaching to the test. Of course one has some personal data on the subjects (date of birth and gender to begin with, and one may appreciate to have things like ethnicity and social economic status), and then one has a classification of each pupil's educational history -- the effect of aspects of this history is what one wants to assess. Information on the pupils' educational history would include at least: what curricular material, what teaching style, what class-level environment, and what homework policy; there will be many other items.
• The study looks at pupils' progress over the years. One first studies the data in order to obtain best predictions for pupils' performance at later times based on their performance at earlier times (and, maybe, on certain extracurricular factors), and then one studies how the addition of data on pedagogical practices modifies these predictions. Basically one asks, does the information that the XYZ pedagogical practice was used change materially the pupils' predicted performance.
• The study must be managed by a group that is independent of any particular educational strategy and any particular set of curricular materials, or the study must be jointly owned by different groups.
• The study must be not more than 20 years old, not less than a few months old, and must be relevant to current educational practices in the United States.

I don't think this is asking for anything unreasonable, given the large total scale of efforts funded by the Federal Government and by various Foundations. Deliberately left out of the above list, because of the many impediments that this would place in the way of the study, is the feature of randomized assignment of pupils to programs. I've looked for (references to or reviews of) studies of the above kind in places such as the following:

- Douglas A. Grouws (Ed.): Handbook of Research on Mathematics Teaching and Learning. A Project of the NCTM. Macmillan Publishing, 1992.

- Alan J. Bishop, Ken Clements, Christine Keitel, Jeremy Kilpatrick and Colette Laborde (Eds.): International Handbook of Mathematics Education. 2 Vols. Kluwer Academic, Dordrecht, 1996.

- Lorna J. Morrow and Margaret J. Kenney (Eds.): The Teaching and Learning of Algorithms in School Mathematics -- 1998 Yearbook. National Council of Teachers of Mathematics (NCTM), 1998.

- A. Sierpinska and J. Kilpatrick (Eds.): Mathematics Education as a Research Domain. 2 Vols. Kluwer Academic, Dordrecht, 1998.

- Anthony E. Kelly and Richard E. Lesh (Eds.): Handbook of Research Design in Mathematics and Science Education. Lawrence Erlbaum Publ., 2000.

- National Research Council: How People Learn--Brain, Mind, Experience and School. Under the responsibility of the Committee on Learning Research and Educational Practice. National Academies Press, revised edition, 2000.

- National Research Council: Adding it Up: Helping Children Learn Mathematics. Edited by Jeremy Kilpatrick, Jane Swafford and Bradford Findell. National Academies Press, 2001.

I've also looked through the Journal for Research in Mathematics Education, the journal Educational Studies in Mathematics, and other such journals, I've asked people of various backgrounds, and I've generally paid attention to writings on K12 mathematics education reform.

Finally I'm persuaded that there is nothing there.

To be sure, there are plenty of efforts in mathematics education research. Many of them provide results that are of anecdotal and perhaps of inspirational value. Many appear to be tightly linked to a particular implementation of some reform, limiting their scientific standing. It really looks as if all the recent United States efforts in education research have not produced a single respected comprehensive study of the kind outlined above, let alone a body of authoritative research that provides firm empirical guidance for mathematics pedagogy.

Fortunately we still have our common sense to guide mathematics education. Unfortunately (but it would take us too far afield to discuss it further here) present trends towards discovery-based learning and constructivist pedagogy seem as little rooted in mathematicians' common sense as they are rooted in education research.

In conclusion I will just mention some other relevant references. The reader looking for a more positive view of mathematics education research can start with the earlier mentioned thick volumes, which do try to put the best face on things. On the other hand, my perspective is in line with "Theories That Gyre and Gimble in the Wabe" by Lynn Arthur Steen, Journal for Research in Mathematics Education, March 1999, pp. 235--241 (a review of one of the mentioned volumes); in "Improving the <<Awful Reputation>> of Education Research" by Gerald E. Sroufe, Educational Researcher, October 1997, pp. 26--28; and in written testimony by R. James Milgram presented before the U.S. House of Representatives Committee on Education and the Workforce, February 2, 2000. As a general reference I mention also "What's At Stake in the K-12 Standards Wars: A Primer for Educational Policy Makers" edited by Sandra Stotsky, Peter Lang Publishing, 2000.

Bas Braams Courant Institute New York University