Of Oversimplification and Education

I just finished reading a piece in The Economist, "Flipping the Classroom". It begins by detailing the laudable work of Sal Khan in Silicon Valley, which is leading to great classroom success in teaching mathematics to children in the 5th-7th grades among local schools in and around Mountain View and Los Altos. The Khan Academy also has great potential for supplementing middle school science instruction, at least. But then the article morphs into a discussion of teacher appraisal systems, and tries to connect Khan Academy results to the latter.

This tendency to seize on a success in a limited span of the academic spectrum with a limited section of the total student population and to then apply the principles discovered to every subject and all students is typical of people from fields outside education when they venture (as they so frequently do) to publish their opinions on how to solve the problems inside American education (I haven't noted this tendency elsewhere). These opinions are almost always, in a word, simpleminded. The Economist is actually quite a sophisticated publication; but to pretend that Khan Academy-style dashboards can properly measure the success of all teachers in all subjects is simple-minded in the extreme. Exhaustively running through all the objections that might legitimately be raised to such a proceeding would pointlessly tire me out; but I will mention just a few.

The central question here is, How should a teacher's professional performance be appraised? I will quote from One World Secondary School's contract for teachers to provide a model for what I support: 

The criteria for assessing this are knowledge and understanding of curricula; planning, teaching, and class management; assessment, recording, reporting, and accountability for student achievement; pastoral and co-curricular support for and involvement with students; and other professional responsibilities, including approach towards professional development as well as interaction and cooperation skills.

Of these five categories of criteria, only one has to do with student performance on assessable work, whether formative or summative, and that is the second ("assessment, recording, reporting, and accountability for student achievement"), and only a portion of this is attributable to what the students do; the vast majority of these assessment criteria are determined by what the teacher does, that is, by the one being assessed (and promoted or fired, in The Economist's vision), not by someone else. Deepening one's understanding of the curricula one teaches, planning for and managing classes, and supporting students outside of class time are also very important professional responsibilities for a school like One World Secondary, and for nearly any other secondary school as well; and yet note the qualification, for I do not believe, in spite of my support for this particular set of appraisal criteria, that it is necessarily the right answer for every school in existence; and therefore our approach to teacher appraisal, and to commenting on educational issues in general, is not so simpleminded.

Lawyering For America

I have been reading with some horror accounts of the Los Angeles Times's decision to publish value-added scores for individual teachers in the Los Angeles Unified School District (my one-time employer), and in particular the story of the suicide of one of the teachers named in the paper with the implication that he was a bad teacher. It's easy to feel, under these circumstances, that the editors of the Times have blood on their hands.

Accountability is a concept that has spread from the business world into education across the English-speaking world for the last two decades, and has become allied to another characteristic of the English-speaking peoples, the tradition of a free press, to produce a marked emphasis on reporting school results. Recently, this has shifted, due to some suspect reasoning that I have questioned in a separate post, from reporting on accountable public schools to reporting on accountable public school educators (especially teachers and principals).

Some make the lazy argument that there is no other profession in which practitioners are not held accountable for their results; but in other professions such accountable results discussions mostly take place in private offices between employees and their immediate supervisors, not in the public press. Such reporting is bad public policy and a disservice to the public. Among other effects, one can well anticipate that it will drive some young people who might be considering teaching as a career into other occupations, and so drive up the cost of education to the public by reducing the supply of teachers. By contrast, I believe that Scotland has an admirable system of educational accountability, which highlights the public posting of inspection reports along with the results on upper secondary examinations that have real consequences for the students, instead of our system foolishly designed with no real incentives to prevent students from making funny face designs on their multiple choice Scantron forms, which used to happen at Locke High School when I first started teaching there (who would do that on an SAT that they had paid for?).


But I have a proposal for those who take extreme positions regarding reporting the results of accountable professionals: let's impose what's being forced on teachers on some other professionals. Failing hospitals don't just produce substandard scores, they kill people, as was believed to be the case in the community surrounding the King-Drew Medical Center in Watts when I was working nearby at Locke High. Perhaps it's not enough that this hospital was shuttered, and only allowed to reopen under different management some years later (a hospital turnaround); why don't we go after its doctors by name, and publicly name in a large-circulation newspaper doctors whose patient mortality rate exceeds the norm anticipated? We can come up with associated terms corresponding to value-added metrics for these bad doctors; perhaps the mortality-added measure of Doctor Murderer? Wouldn't that have a salutary effect upon the medical community? 

Or how about lawyers? They can kill too, instead of just producing disappointing results on nearly meaningless tests; culpable negligence on the part of a defence lawyer, for example, can lead to innocent human beings being put to death in countries practicing capital punishment. So they too might have their mortality-added measures calculated; or perhaps other metrics for allowing criminals to escape conviction (this would let us calculate, and publicly report, a prosecutor's crime-added metric), or for false incarceration, or contribution to general injustice.

Of course, if we publicly humiliated other professionals in this way, we would probably hurt education, since we can anticipate that the supply of young people willing to subject themselves to this form of shaming would be diminshed, and a decrease in supply would drive up the cost of a doctor's or lawyer's labours in a free market economy; and a major reason why American schools are so thinly staffed, in terms of student-teacher ratios, is because American teachers are relatively expensive compared to their peers overseas, but still feel underpaid since their salaries are so low compared to the sky-high costs of American professionals in other careers like law and medicine.

But I can think of another solution for that, as well: let's introduce new, alternative certification programmes for law and medicine, programmes that would increase the supply of the practitioners, and thus drive down the wages, of these professions by taking recent college grads, giving them six weeks of supervised summer training in hospitals and law firms, and then unleashing these ambitious young people on the public.

One name comes immediately to mind: anyone want to sign up for Lawyering For America?

The doctors' equivalent forces me to think a bit longer . . . what do you think of Meds For America?

A Seeming Paradox Regarding the Use of Time in Education

American politicians--mayors, governors, presidents--have made a fashion lately of calling for longer school days and years, and with some reason: high performing charter school networks such as KIPP (the Knowledge in Power Program) depend upon longer school days, Saturday schooling, and brief summer sessions to maintain their impressive results. But where the money would come from to pay for this increase in schooling is unclear, and the evidence on the relationship, if any, between the length of the school day and year and consequent student success is also far less clear than is often assumed by those whose knowledge of education stops at our shores.

For example, Eurydice, the educational knowledge network and database for Europe, studied the widely varying uses of time among the 37 nations belonging to it and found no correlation at all between the length of the school day and year and student achievement on measures like PISA, the Programme for International Student Assessment of the OECD. And two of the top performing educational systems with which I am familiar, those of Finland and South Korea, also provide counter-evidence to the simplistic thesis that more time in school necessarily leads to more learning. In Finland, students don't start school until age seven, and they spend fewer annual hours in school than any other country in Europe, yet their test scores are ahead of those of all other European (and American) countries by the time the students are 15. And in South Korea, while the school year is longer than the American and tops 200 days per year, the annual number of hours teachers are scheduled to be in class is the lowest of any country in the OECD.

What to make of this data? The key point, I think, is that the question of how students use their time is not equivalent to the question of how long they are in school. This is where socioeconomic status, and in particular the culture of the home, prove vital. European citizens with rich cultural offerings and thorough social safety nets offer their young a wealth of activities outside of school time, which some of them are wise enough to enjoy (not all do). Koreans and other east Asians, especially in their teen years, spend long hours attending cram schools or being tutored after school, and it is this use of time, rather than anything special happening in their schools, that leads to their dominating the international test score league tables. Wealthy Americans have their children in numerous activities, especially sports, after school and on weekends, and some have the means of taking their children on long summer vacations which can be very culturally enriching in their own way. But poor Americans in ghettos have virtually nothing to do all summer long but watch TV or get in trouble with local gangs.

This is where equity issues get tricky. On the one hand, spending more on the schooling of some students than on others in order to buy the former extra time with teachers feels innately wrong and unjust because it violates the principle that all people deserve equal treatment. On the other hand, in a Western country with such gross social inequities as the United States, it is patent that if we don't intervene in schools to try to compensate for the inequities elsewhere in our social system, we are confirming those inequities, and dooming most (a few of outstanding talent will escape the ghetto anyway thanks to our meritocratic opportunities) poor children to second-rate lives.

Utilizing the approach to social justice experimentally derived in Marc Hauser's Moral Minds, which implies that we make equal distribution of social goods whenever we can and prioritize giving to the underprivileged first whenever we don't have enough of some good or service to go around, I think summer schooling for all a sound if, for now, utopian social policy, and suggest that we begin by lengthening the school year of our most underprivileged first, and gradually lengthen it for everyone.

On the Exetasecrats

I just coined that word--I derive it from (also my invention) exetasecracy, "rule by examination".

Jay Mathews did me the honor of quoting me in The Washington Post about a month ago, and specifically my phrase "government of the people by the highly educated for the well placed vested interests". Here I am drawing attention to the middle of those three prepositional phrases. In contemporary American civilization, a highly educated person is someone who will have taken many tests, and have done exceptionally well on them. Democrats, Republicans, talking heads supporting education reform and those opposing it, virtually all (including me) of those participating in any of our education debates, now that the Bush Administration is out of power, are exetasecrats, people who did well on tests when they were in school, and, because much of their self-validation derives from their performance on those tests, believers in their validity and usefulness.   

But this post continues (and will probably conclude, for now, since I am actually a fan of TFA, on balance) my attack on Teach For America, begun two posts back and now in its third installment. TFA, being highly selective, is filled with bright young exetasecrats who are setting up an occupational sideline in education while they are between tests (LSAT, MCAT, GMAT, exetasecrat). Having amassed two or three years in the classroom before they moved on to other jobs, possibly involving telling the rest of us how to rearrange the educational system, they are likely to be very thinly informed, from a research standpoint, on the possibilities and limitations of various assessment systems; but because they are intelligent, they are likely, especially if they did their teaching in primary schools, to recognize that even our typical substandard state tests are more defensible, from a pedagogical standpoint, than the ludicrous high jinks that their older colleagues sometimes come up with and impose upon the public (I had a colleague once, a dear lady, whose idea of an appropriate final exam for her summer school English class was an African marketplace--not that she had ever been to a real one [not much English in the one I went to]).

But when one has experience, perhaps from teaching high school or college English, in working with more rigorous and admirable testing systems (I am thinking of those from Cambridge Assessment and the International Baccalaureate, but there are others), which do not reduce the complexities of human intellectual activity in the extreme way required for American educational data systems to be of any use--the reduced, lost information and truncated discourse of the machine--then one is far less enthused by all the simplistic prattle accompanying proposals to base our children's education more and more squarely and narrowly on the very imperfect assessment instruments present in the United States; and this is what the conventional proposals of well meaning amateurs, who have made endless money from selling pseudo-thinking machines, and who believe sincerely in the power of those machines and have seen them work well in other fields and whose own self-validation has come largely in connection with those machines, amount to.

Tony Wagner of Harvard appropriately suggests, on page 268 of The Global Achievement Gap, "Instituting better assessments is the one most important change we could make tomorrow that would have the greatest impact." Howard Wainer, formerly chief statistician of the Educational Testing Service who now teaches statistics for the Wharton School, has just published, via the Princeton University Press, Uneducated Guesses, which reveals how little our exetosecrats understand about the instruments that have provided them with important boosts in their young careers and which they now promote, to the impoverishment of us all.

On Test Scores and Teacher Appraisal

I believe in performance pay, and have had bonuses based on teacher performance built into One World's planning ever since our first charter petition. In fact, following the lead of European Schooling Helsinki, one of the model schools with whom we have established contact, a teacher's appraisal is proposed to multiply that teacher's base salary by 0%-48%. So it may be surprising that I object to the notion that anything approaching 50% of a teacher's appraisal be based on that teacher's students' scores on annual state tests like the California Standards Tests; but I believe that any such proposal obviously betokens a greatly oversimplified view of teachers' work and an excessive emphasis on grossly inferior tests, the over-reliance on which being already responsible for this dark period in our educational history.

Two related questions are raised in this regard: how are teachers to have their performance appraised, and who should do it?

In an early draft of a performance pay formula for our school, I included a value-added model in our proposal, and the value added to students' actual vs. predicted performances on external assessments was slated to be worth 20% of a teacher's appraisal, which would have determined that teacher's bonus; but I now think that 20% was too high. The model I currently support, after having engaged in international correspondence with some of the world's best qualified experts on these matters, utilizes five criteria to define professional performance: "knowledge and understanding of curricula; planning, teaching, and class management; assessment, recording, reporting, and accountability for student achievement; pastoral and co-curricular support for and involvement with students; and other professional responsibilities, including approach towards professional development as well as interaction and cooperation skills." "Accountability for student achievement" does refer to that teacher's students' scores on external exams; but that is only one of four items listed in that assessment criterion, which, if the criteria were equally weighted, would combined make up 20% of the total (but I no longer support any such addition, because it is an instancing of the fallacy of composition, which assumes that a whole is equal to the sum of its parts, which is invalid). 

In short, I do not believe that any mathematical formula can perfectly appraise a teacher's performance; and even if I did believe in such a formula, to base at least half of x's appraisal on the performance of y, not x, is unjust; and because a group of vengeful students could potentially sabotage a teacher's career by intentionally spoiling their exams, which carry no consequences for them (this is the worst of the many flaws in America's test-based accountability system; and the intentional spoiling of tests was a reality at Locke High School when I first started there ten years ago), such proposals, which have recently been enacted into law in several states, make for very unsound public policy.

And this brings us to the second question raised above, that of who should be carrying out teacher appraisals. For in fact I have above quoted from contract language for our school--and while I believe it to be consistent with our vision and aims, I do not pretend to claim that it is (or isn't) appropriate for other schools, especially for very different ones, such as primary schools. My point is that no teacher appraisal formula is likely to work well for every school in any state; and if our educational leaders really want to serve (not lead, or rule, which can run counter to the principles of democratic education, and does so in this instance) their schools and their students well, they would do best to take their hands off any such decisions, and instead devolve that power further down to a level closer or equal to that of the individual school, and should forgo trying to save humanity by dictating teacher evaluation formulas from their inner sanctums in the hopes of finally finding the magical secrets needed to save all those profane souls not similarly blessed but instead consigned to schools under their power.

Of Democratic Education as a Result of Federalism Rather Than of Political Advocacy

In democratic education, as much power as possible devolves to the student, which means as little as possible is reserved by those higher up. At the extreme, this would be the situation of the Wild Boy of Aveyron, surely not the ideal we seek; but pursued to its optimal, not its extreme, extent, this should result in proper empowerment for everyone from the President of the United States all the way down to first grade special needs students. 

But this is by no means the exclusive conception of democratic education. A numerous breed at the present time, the progeny of Wendy Kopp's senior thesis at Princeton, are indoctrinated at a suitably malleable age to undergo 5-6 weeks of gruelling summer training, then march off into tough neighborhoods for two years of teaching service for America. Some of these talented, idealistic young people continue teaching; many of the others (too many, I think) go on to law school or some similarly higher status profession, but pledge to "remain involved" in education in some manner, hoping to continue working to change America's very unequal educational system. These latter more or less by definition lack the experience and knowledge to valuably contribute to our education debates, but they remain passionate and fired up, and inexperience doesn't stop them.

A surprisingly large number of these reform-minded young people are able to insert themselves into positions of considerable influence and power, and many of them are now engaged in education advocacy, usually for policy positions consistent with the doctrines underlying Wendy Kopp's original vision. These TFAers are generally talented, enthusiastic, hard-working, and naive, and, because of their relative lack of experience and relevant knowledge, are passionately advocating for ill-founded positions that may be doing more harm than good, as I will proceed to demonstrate in upcoming posts.

I will begin by attacking the proposal that at least 50 percent of a teacher's annual performance evaluation should be based on that teacher's students' performances on required state testing.     

On Excellent Teaching

Rafe Esquith might be the best teacher in Los Angeles. In any case, he is very likely the only one to have been awarded an honorary membership in the British Empire! So it's important for Angelenos, and for those who seek to honour teachers, that his fate not end up like that of Jaime Escalante.

Escalante, you may recall, was the amazing math teacher at Garfield High School in east Los Angeles who got so many students to pass AP Calculus that the College Board couldn't believe it and forced them all to resit the test, which they did under external scrutiny and performed even better than they had the first time. His success was detailed in a biography by Jay Mathews of The Washington Post that led to Stand and Deliver, a major Hollywood feature. But colleagues grew jealous, and new administrators put pressure on Escalante to conform to their demands, so the non-conformist Escalante left Los Angeles, and Garfield's math program sank back into mediocrity.

Now an energetic new superintendent in Los Angeles is seeking and finding support for a series of well intended reforms, including concerning teacher appraisal. In the name of equity, most such appraisal systems introduce a single set of measures, sometimes quite complex, to appraise all teachers, even throughout a huge and complex system like that of the Los Angeles Unified School District. But there are pitfalls to be avoided.

One might "measure" (the meaning of this word has been metaphorically extended, let's remember) teaching by outputs such as results on external examinations, for example. But this can lead to ridiculous results. My son Ryan, for example, might be given a math test and achieve an advanced rating, and his teacher, whose class he just entered last Thursday, might be given some kind of award. But what the principal or those higher up might not know is that my wife, a math teacher, taught him the whole of sixth grade math during the summer, and is now starting him on algebra (Ryan's still ten), because we would never allow the California (or any other American) math curriculum to control our children's mathematical education. Would you say that his sixth grade teacher at school deserves the credit for his mathematical success? And if I were a teacher being appraised in this manner, my first move would be to connect my students' parents with a good tutoring service; this would likely be a much more profitable use of my time than changing anything I was already doing in class, assuming I was already doing the best I knew.

Okay, so let's not measure teachers by their students' results (which at one level feels unjust anyway: we don't normally judge x on the basis of y's actions). Let's go into the classrooms and judge our teachers by their own, rather than their students', actions. After all, certain behaviors by teachers have been linked with better student performances (usually on tests--and when those tests are quite narrow in subject matter and question type, the advisability of this procedure is greatly diminished), so we might score them highly for demonstrating such behaviors during their teaching.

But hold on, what about Esquith and Escalante, two Es of excellence, both brilliant teachers with long records of outstanding results produced, as nearly as we can tell, through their own rather than someone else's efforts--won't we be in danger of punishing unorthodox, brilliant teaching? Both of these men have been known to be scornful of the standard pedagogical dicta that were attempted forced on them by L.A. Unified administrators; are we sure that we're not about to fall into the same trap here? Can the newly empowered administrators be so confident that they have found the perfect formula to assess all teaching in all grade levels and subjects, even as different as fifth grade classroom teaching and Advanced Placement Calculus, with fairness and accuracy? If they have found such a magic formula, let them produce the evidence they have it; and if they do, why haven't they been applying this special virtue to better students' lives in all venues where they've been before? Perhaps the discovery is new, and has had only limited trials; fair enough, but then let's dial down the confidence factor, and be more humble and honest about the risks involved. And if they don't, we're all at risk.

Of course, doing nothing in such a desperate district as L.A. Unified would be even riskier for hundreds of thousands of students; but I wouldn't want to see a brilliant teacher like Rafe Esquith, or others like him, depart like Jaime Escalante.