Showing posts with label School systems. Show all posts
Showing posts with label School systems. Show all posts

Sunday, October 6, 2013

WYSIATI and the Assessment of School Systems

For me, it seems like Amanda Ripley is everyplace right now. This morning, when I went out for breakfast, I carried a copy of The Atlantic with me into Panera and consumed her cover story "The Case Against High-School Sports" along with a pastry and a latte. In the afternoon, I listened to a radio interview she recorded with Bloomberg Radio. And tonight I finished chapter 3 of her book The Smartest Kids in the World. I'm finding this increased exposure of comparative education, a passion of mine, stimulating, and in spite of some critical remarks I have been making here and there (most notably in a review that was published in August by Whitney Tilson, in one of his periodical newsletters and on his blog, of her Wall Street Journal article "The $4 Million Teacher"), I am enjoying all this Ripley, believe it or not.

Nonetheless, her dismissive evaluation of Switzerland's education system, which I think the best in the world, in the Bloomberg interview and her valorizing of the Korean and Polish school systems on account of their improvement shown in the results of PISA (the Programme for International Student Assessment, the best known international tests that allow for comparison between different countries' educational systems) is indicative of an especially problematic instance of WYSIATI, a cognitive malady diagnosed by Daniel Kahneman and especially prevalent in educational evaluation, to the great harm of us all.   

A couple of years ago I read Thinking Fast and Slow by Daniel Kahneman, emeritus professor of psychology at Princeton and, most unusual for a psychologist, winner of the Nobel Prize in Economics. Among the fascinating chapters in that book is chapter 7, "A Machine for Jumping to Conclusions", which introduces readers to WYSIATI, which stands for "What You See Is All There Is", the cognitive fallacy that is the "machine" referred to in the chapter title. The fallacy is well illustrated by a cartoon I saw once, which shows a man searching desperately under a streetlight at night for his glasses, which have fallen off. When asked why he has been searching in the same place for so long rather than looking elsewhere, the man responds, "Because under the light is the only place I can see." Similarly, when asked, "What's the best education system?" even well-informed researchers in comparative education often engage in question-substitution, another habit of our inherently lazy minds, and convert that question into the much more easily answered "Which country [or school, or teacher, or student] has the best test scores?" and then provide the answer to that in substitution for the answer to the original question, whose answer would necessarily require the human use of qualitative judgement in addition to the machine-like use of memory for relevant quantitative data.

Too many pundits convert the hard question, "Which countries, and which aspects of their systems, might serve as useful models for the United States [or another country] in improving our educational [or health care, or other] system?" into the more easily answered, "Which countries [or states, districts, schools, teachers, students] have better scores than ours?" Those of us with first-hand experience in such jurisdictions are well aware of the tradeoffs required for higher test scores, and avoid publishing such simplistic evaluations, even if experts with power, influence, and overconfidence (most prominently the PISA datameister at the OECD, Andreas Schleicher) are regularly provided with audiences for broadcasts of similar spurious claims and arguments.

Sunday, March 24, 2013

A One World System of Schools

I am still working to open a school for my son, and for all like-minded families with children. I currently propose to open, with my colleagues, an upper secondary school, a lyceum (an anglicization of lycee, the nearest extant equivalent to the kind of school we propose), providing education for the 10th-12th grades, in 2015 (it usually takes two years of preparation to open a good school, and although we've been preparing for longer than that, we want to be great). This first one will be an independent school, since I've given up (for now) the idea of finding an American school board with enough courage to back our charter, which is ready to be put into operation and is replicable at currently available public school dollars.

In a One World system of schools, our three-year lyceum would be preceded by a pair of three-year middle schools, a boys' school and a girls' school, which might more plausibly be established as chartered schools; for there is plenty of evidence available (a) that it is in the middle school years when Western students fall significantly behind their peers in east Asia, and (b) it is the premature obsession with the opposite sex during early adolescence that accounts for a good deal of (a). And of course it is unfortunately well known that the early preoccupation with dating and social relationships not only detracts from learning, but also all too often leads to teenage pregnancies and thereby cyclical generations of wrecked lives. Better to keep the boys and girls away from each other in these years; whereas our lyceum and primary school would be coeducational, I advocate single-sex education during the middle school years (which are, for us, grades 7-9; we use the term middle school here because it is an exact translation for these institutions in the Chinese, Japanese, and Korean languages, which, in relatively reformed and modern versions, constitute the nearest equivalent to a model for us to use during these years).

And the lyceum and middle schools should be preceded by a six-year primary school, for children aged 6-11, of both sexes in the same classrooms. These might ideally be government schools, and their successful establishment is crucial to the democratic state, as is in reality the whole of comprehensive schooling, which should last for nine years, ages 6-15, as it does in Switzerland, which has the best overall school system, as I have argued elsewhere. Our model for primary school is Finland, whose primary schools are not very different from the fine elementary schools here in Irvine, except for the fact that the Finns, like other Europeans, begin learning a second language early, from the first grade onwards, as I would like to see Americans do as well: second language learning from the first grade would be the principal novel (for America) feature of a One World Primary School.

The ages and grades we have defined for primary, lower secondary, and upper secondary education, which are first-sixth grades for children aged 6-11, seventh-ninth for students aged 12-14, and the tenth-twelfth grades for students aged 15-17 (with a postgraduate year available to students who might benefit thereby, therefore taking education for some students up to age 19), are the most commonly used in the world, and therefore using them allows us to make maximum use of the experiences of all of the people of our planet as we strive together to raise better people for our world.

Sunday, August 14, 2011

Why the Norwegian School System Might Be the World's Best

I'm not sure there's any such thing as a single best system, but I do go looking for models, and have come to appreciate Norway's. This is a follow-up to my previous post in which I focused on statistical methods by which we might approach this question.

Average student attainment Norway ranks high here, behind only Australia and New Zealand, and is number one in mean years of schooling already achieved, at 12.6, for students aged 25-34, according to UNESCO. (For Americans who might be wondering, we rank third, at 12.4, with New Zealand just ahead at 12.5.) The UN's Institute for Statistics calculates that currently enrolled Norwegian students are expected to push this way up, to 17.3, and this rate of increase is something that should give Americans pause, as the generation of American children in school is projected to no longer rank in the top dozen if current trends continue.

Average student achievement Norway is a thoroughly middling country in its PISA scores, having, like the United States, a most recent median PISA score of 500. Norway's relative scores have been rising recently, unlike many other Western countries, perhaps in response to reforms to more highly value knowledge. (To give examples of how anti-competitive Norway's approach to education has been in the past, I'll point out that Norwegian children normally receive no grades at all during primary school, and even their secondary school leaving exams, typically taken at age 19, only require the sitting of one examination, in an advanced subject that the student will have chosen well in advance.)

Approximate knowledge and skills of the average young school leaver Multiplying attainment by achievement scores, Norway still holds on to the fourth position, behind New Zealand, Australia, and Korea. The USA, for those interested, remains out of the top dozen, behind also, in alphabetical order, Canada, Estonia, Finland, Germany, Iceland, Ireland, Japan, and the Netherlands.

Employability of non-college-bound students Individual transition plans are very important for special needs students leaving formal education for the "real world". Norway has worked hard in recent years to upgrade these, which appear to be relatively effective: 45% of former special needs students are employed in Norway, which is almost the highest figure internationally (Switzerland's is 52%). Norway has also been moving towards apprenticeships, a German best practice, to help its students who are unenthusiastic about higher education, and has also started return-to-training initiatives that have proved relatively effective. Possibly as a result of these various reforms (the existence of plenty of North Sea petroleum doesn't hurt), Norway's current unemployment rate of 3.4% is almost the lowest in the Western world. Wouldn't Barack Obama like to have a number like that right now?

High-achieving universities Norway is undistinguished here, perhaps because its secondary school leavers have not been rigorously prepared (I think this the greatest weakness of the Norwegian system; commentators recently estimated that, on PISA, high-achieving Norwegian math students were three years behind their counterparts in Shanghai). The top-ranked Norwegian university appears to be the University of Oslo, #75 in ARWU and #100 according to QS; THES has it ranked below the University of Bergen, which is only #135. (Again, I don't trust these numbers to be all that precise, no offence intended towards their creators, but I do think they are generally indicative of trends.)

On balance, what are the strengths of the Norwegian system, and what are the trade-offs? Perhaps a key strength to Norwegian and Nordic education in general is not to be in a rush with children. The Norwegians require ten years of comprehensive education under their slogan "One School for All", from ages 6-16 (they start a year earlier than some of their more admired Nordic neighbors--researchers have concluded that starting school too late correlates with lower test scores); this is consistent with their goal of "a high general level of education in the entire population", which I assert (a) they are achieving more than any other nation and (b) is a worthwhile goal to pursue, rather than the highest test scores. This may be a trade-off, since the fastest way to raise test scores is to kick out (to where?) all the low scorers, and since it is difficult for any impatient would-be miracle worker of a politician or superintendent to claim credit for miraculous rises in attainment the way they can for miraculous rises in test scores.

Also, the Norwegian system intelligently provides a place for everyone, for all children and adults, with their various talents and ability levels, in accordance with a vision of a society "where citizens master the art of living together"; this may be contrasted with a society and government of the people by the highly educated for the well placed vested interests, which is what I fear American society has become. The only trade-off I see to pursuing such a vision is that it will upset entrenched interests and also those who love America and want to patriotically believe, as a default position, that our customary way of doing things is automatically the right one, in spite of considerable evidence to the contrary. It's hard to change systems fundamentally, and requires a lot of patience and persuasion, as I am attempting to use.

I also commend Norwegians and others in northern Europe (including, and especially, the new government of the United Kingdom) for their legislation allowing the public funding of privately managed schools, since both of which factors, as independent variables, are associated worldwide with greater student success.

And this brings me to the chief way to proceed with this kind of information, easily available via the knowledge network of the Information Age.  I think we should (1) start with a good general system design, like Norway's, and then (2) make it better by seeking best practices as discovered by researchers investigating independent variables, (3) remembering always to check the internal consistency of these attempted improvements, since in reality variables do not operate independently, but are embedded in systems in the real world lives of students and teachers.

The Best Schools in the World

A couple of years ago, when I was busy with refining the plans for One World School (an earlier conceptualization of One World Secondary School), I would have answered this question with model independent schools that were inspiring and still inspire me (the International School of Geneva, the United World College of South East Asia, others). Now, however, like others, I have shifted my focus away from individual great schools (one of which I am still hoping to start) to great school systems.

Professor Linda Darling-Hammond, in her The Flat World and Education, nominates Finland, Korea, and Singapore as having systems that America can learn from, and many other commentators (including, influentially, McKinsey & Company) have named these same three as having perhaps the best systems; but none of them would make my top three.

Any ranking (I don't believe in precise numerical rankings, but do believe we can learn from others) of educational systems will depend upon its criteria to generate output. What criteria should be used in ranking school systems?

Most commentators on education depend upon test scores (I promised in an earlier post to return to this theme), the most influential internationally being those of PISA (the OECD's Programme for International Student Assessment), but that's not where I would start. In the final analysis, I don't care as much about how many multiple choice questions on reading, mathematics and science that a student got right at the age of 15 as I do about how well educated that person will be as an adult, after having left school, with fifty or more years of life left to live. There is no precise way to measure how well educated someone is (a point to be borne in mind by those obsessed with measurement in a field where much of importance is exceedingly difficult to measure); the best statistical proxy is, I think, attainment, which generally refers to the number of years of schooling successfully completed. (For example, in my case, having studied [in school--I hope it's clear I've done a lot of studying since then] one year past a bachelor's degree, that number is 17.)

There are two statistics for attainment we should care about, one for students in school today and another for those who have already finished school. I care more about the future, and so give a bit more weight to the statistic for those youngsters currently in school than for those slightly older who are currently out of it, but both are important, since the former numbers are, if well reasoned, nonetheless speculative, and the latter, real numbers for recent graduates will continue to be important for a long time, with many decades worth of elections upcoming for people to vote well in or badly (or perhaps, even, to vote badly twice in a row!). I don't give weight to the attainment of older people, which hurts any U.S. ranking, since we were for a long time number one in the world in educational attainment but have slipped back in recent years as other countries have been improving their educational systems faster than we've been improving ours.

Adding together these figures for expected years of schooling successfully completed (easily available from UNESCO's Institute for Statistics) yields a set of figures useful for estimating educational attainment. But we also care about student achievement--how much a student learns in any given year of schooling--as a measure of the productivity of the school system. We might actually be able to multiply this productivity by the total anticipated attainment to give us a better measure of how well educated today's young people are going to be as they go through their lives, are productive (or not), and make important decisions. This is where the PISA scores become useful. I have multiplied the most recent median PISA score for each country available by the above estimate of anticipated attainment to estimate how well educated the average future citizen of each of these countries will be, not taking the numbers too seriously but becoming convinced that I was on to some useful estimates and methodologies. The top countries were New Zealand, Australia, and Korea. Only the latter is commonly mentioned as having a really top-ranking educational system, since most of these assessments depend more upon test scores than upon attainment. I will return to this theme later, including taking the time to debunk some myths about which top-scoring systems we should emulate.

But these figures only estimate how well educated the average future citizen will be. How about those at the extremes?

I believe the basic humanity of a country can be judged, in part, by how well it treats its least fortunate and neediest citizens. In addition, a law of classical economics, that of comparative advantage, teaches us to help our weakest citizens to become productive first if we want to maximize the gain for society as a whole.

The students most in need of special protection, it seems to me, are children with special needs, which every country will always have. A clear sign of treating such students well is making sure that they are gainfully (and, it is hoped, happily) employed when they become adults. I looked for employment statistics for people who had been students with special needs; these were very difficult to find, but I found some, along with other information on youth unemployment and the reputations of various countries' laws and programmes for those with special needs, and made some adjustments to my rough rankings, having at this point to give up any pretence of a formula.

Finally, those who are best off, educationally speaking, are those bright students commonly called gifted. They are inherently important, as all human beings are, but especially because of their impact in generating original ideas that often lead to jobs for everyone else. They also have special needs, and are often well served by programmes designed with their needs in mind, which allow them the opportunity to progress at their own pace; and they often desire to end up in the kind of prestigious, highly resourced universities that dominate the international ranking systems. The three most widely used international university ranking systems are those of the Times Higher Education Supplement, the Academic Ranking of World Universities, and the Top Universities ranking of Quacquarelli Symonds (QS). American universities dominate the top of all three. I made adjustments to my rough rankings, and pushed up the placement of the United States many steps on this basis. Other countries with relatively highly ranked universities also moved up, including England, Canada, Japan, and Switzerland.

This, I feel, is a pretty good way to approach assessing the overall quality of educational systems. Of course, others with more time or better resources could probably elaborate on it and improve it; but using this approach is useful, I believe, in helping to debunk various myths or shallow platitudes regarding which educational systems we all ought to be looking at as models (I expect to return to that topic at some point in the future).

The results? Of the three oft-cited countries mentioned earlier in this article, Korea did the best, but I found three other countries I would recommend before it. I have already mentioned two, Australia and New Zealand, which UNESCO says lead the world in expected years of schooling for those currently students; but I have one placed above them, and I will elaborate upon this country's system and various other implications of this study in another post, since this one has already gone on long enough. Therefore, the (surprise) winner, in this little competition to determine the world's best school system?

Norway.