Teacher effectiveness and student test scores

Last week-end I met an Ohio teacher named Judy who works with elementary-aged, struggling readers. I asked Judy where she taught and, as she told me, she couldn’t help sharing her intense frustration with Ohio’s system for evaluating teachers which, according to her, relies heavily on test scores. She felt this system extremely unfair to her since, almost by definition, the students she worked with will make only limited progress over the course of a school year. The pressures of these accountability measures were clearly robbing Judy of the pleasures of teaching.

Implicit in NCTQ’s assertion that university-based teacher prep programs aren’t adequately preparing teachers to teach reading is the claim that teachers are generally unprepared for the rigors of teaching. The sense that many teachers aren’t up to the task is behind the spread of teacher evaluation schemes that Judy found so stressful and unfair. Most of these teacher evaluation systems rely on student test scores which may seem logical but, as it happens, student test scores are not a fair or principled way to evaluate teachers. There are at least a couple of reasons for this. First of all – and this may come as a surprise to many people – research indicates that teachers are NOT the most significant influence on academic achievement. According to Berliner and Glass (2014), most research indicates that less than 30% of a student’s academic success is attributable to schools, and teachers are only part of the overall school effect, perhaps not even the most important part. Other factors, particularly the material effects of poverty, account for over 60% of the variance that can be accounted for in student achievement (Berliner & Glass, 2014). Another problem with using test scores to measure teacher effectiveness is that these measures tend not to be very reliable. Teachers whose students do very well one year often do not do well the following year (Berliner and Glass, 2014), not a surprising finding given the influence of non-school factors in student achievement.  

I am not arguing that teachers should not be evaluated. Far from it. But teacher evaluation schemes based on test scores are unfair, unreliable and ineffective. And they make a difficult job – teaching – even more difficult.

Reading pedagogy will never be sufficient for overcoming the material effects of poverty

NCTQ’s attacks on teacher education are animated, in part, by the belief that university-based reading educators are mostly whole language advocates who eschew the teaching of the fundamentals of reading (i.e., phonics). NCTQ shares this belief with many critics of American education who seem convinced that the emphasis on literature and meaning making in schools of education represents an outright rejection of phonics instruction. This “anti-phonics” stance is seen to be responsible for the poor literacy achievement of American school children compared to children in other countries around the globe. To put the argument more simply, American students are not learning to read as effectively as their peers in other parts of the world and, ultimately, the blame for this situation lies with university-based reading educators who, because of their rejection of explicit phonics instruction, do not adequately prepare teachers to teach reading. However,

·      There is no evidence that phonics has been deemphasized in schools and classrooms across the county. Indeed, spurred by Reading First grants there has been an increased emphasis on the so-called fundamentals of reading, especially in low-achieving, high-poverty schools.

·      There is no evidence that reading education in schools of education is dominated by holistic approaches to reading as many critics have claimed.

·      Meaning-centered approaches (like “whole language”) are not anti-phonics. Letter-sound information is always part of the data readers use to bring sense to texts. After all, no one reads with her/his eyes closed. Meaning-centered reading educators believe, however, that phonics is best learned in contexts where readers use phonetic cues simultaneously with other contextual information to make sense of what they’re reading. These educators don’t reject phonics but do reject isolated skill instruction removed from the context of reading authentic, whole texts. Even the report of the National Reading Panel that the NCTQ has valorized suggests that phonics instruction must be part of a rich and varied program of reading real texts.

·      American schools are not failing compared to schools in other countries. David Berliner’s analysis of international comparisons clearly shows that American students in relatively affluent schools do very well compared to their peers in other countries. The problem is that students attending schools with higher proportions of students living in poverty do much less well compared both to students in other countries and their peers in more affluent schools in this country.

The arguments about reading instruction in the US have focused mainly on pedagogy but the evidence indicates that, while pedagogy matters, it will always be insufficient for overcoming the material effects of poverty. If we are serious about “leaving no child behind,” we must give serious attention to the high levels of poverty among American school children.

What motivates NCTQ?

On its website, NCTQ positions itself as a champion of education whose only interest is improving the quality of classroom teachers. Its history of trashing teacher education in the US based on shoddy methodology suggests another motive: destroying public confidence in teacher training institutions. Indeed, according to educational historian and former Trustee of the conservative Fordham Foundation DianeRavitch, NCTQ was founded by the conservative Fordham Foundation “with the explicit purpose of harassing institutions of teacher education.” NCTQ appears to be in sync with Reid Lyons, the man often described at George W. Bush’s “reading czar,” who once commented that, "If there was any piece of legislation that I could pass it would be to blow up colleges of education." This begs the question as to why NCTQ would be interested in harassing (or, perhaps, “blowing up”) schools of education. The answer lies in the broader efforts of educational reformers to “re-form” education through the application of the principles of the free market. Vouchers, charter schools and alternative teacher education programs are viewed as panaceas for all the problems that plague American education. The basic assumption is that competition insures that only the best schools (including schools of education) will survive. To date, vouchers and charter schools have a very mixed record and, in any case, education based on the free market is fundamentally at odds with the assumption that one of the primary goals of public education is to prepare citizens ready to take their place in a participatory democracy. Likewise, low budget, highly circumscribed alternative teacher preparation programs prepare teachers ready to teach only highly-scripted curricula but who are unprepared to address the broader goals of democratic schooling. In this formulation teachers and students are all seen as mere cogs in the free market that makes little accommodation to the individual differences that make students and teachers interesting, thoughtful people. This dystopian vision of schooling is a threat to democratic schools -- and democratic governance more generally -- and must be resisted. 

In Defense of Teacher Educators

This blog is intended to build on arguments I made in my recently published book, Preparing the Nation's Teachers to Teach Reading: A Manifesto in Defense of "Teacher Educators like me." In this volume, I took on the various reports of the National Council on Teacher Quality (NCTQ) which claim to show that US colleges of education are doing a poor job of preparing teachers to teach early reading. This conclusion is based upon NCTQ's review of course syllabi and textbooks in reading methods courses in teacher preparation programs across the US. Specifically, NCTQ concluded that teacher educators are failing to address the fundamental areas of reading identified by the National Reading Panel. The NCTQ did find, however, that schools of education in some states were adequately preparing future teachers to teach reading. All of the teacher preparation programs reviewed in the state of Louisiana, for instance, met NCTQ's criteria for teaching early reading. Oklahoma, Mississippi, West Virginia and the District of Columbia were among other jurisdictions that did relatively well on NCTQ's rankings. My home state of Massachusetts was among the poorest performers on NCTQ's assessment of how well teachers were being prepared to teach reading. Teachers education programs in Connecticut, New Hampshire, Vermont and New Jersey were also judged harshly by NCTQ. The problem for NCTQ is that, according to data from the National Assessment of Educational Progress (NAEP), 4th grades students in MA, CT, NH, VT and NJ do much better on the NAEP reading assessment than students from LA, MS, WV and DC. To further explore the relationship between student reading achievement and NCTQ's assessment of teacher preparation programs in the various states, I computed a correlation between the proportion of teacher preparation programs that met NCTQ's standards in each state and the District of Columbia and NAEP scores for 4th grade reading in each state. The result was a slight negative correlation between NCTQ's ratings and NAEP scores for each jurisdiction. Put differently, there is absolutely no relationship between NCTQ's ratings for teacher preparation programs in states and the District of Columbia and 4th grade students' reading achievement in those states. If there was any validity to NCTQ's ratings of how well teacher preparation programs in different states prepared teachers to teach reading you'd expect states in which NCTQ judged teachers to be well prepared (like Louisiana) to do well on the NAEP and states where teachers were judged to be less well prepared (like Massachusetts) to less well. But that is not the case and, in fact, there is a slight tendency for states that do relatively well by NCTQ's criteria to do poorly on the NAEP. This finding, combined with a host of methodological problems that plague the NCTQ review (no evidence that syllabi are a valid way to assess course quality, for example), completely undermine any claims made by NCTQ about how well teachers are prepared to teach early reading. My analysis doesn't prove that all is well in teacher education and I'm sure there is plenty of room for improvement (and, no doubt, some programs are not doing as good a job as others). The NCTQ report, however, offers nothing to judge either way.