Commentaries (Susan Ohanian Speaks Out)

Congressman Miller seems to have forgotten that economic cycles have come and gone in the past independent of what 4th graders were doing in math and science. The economies of developed nations will continue to rise in fall independent of test scores. Japan, with some of the highest scoring students in the world, has been in the economic doldrums for almost 20 years. Iceland, with high scoring students, has become an overnight basket case with national debt equal to 850% of its GDP.

Highlighted by cburell

Kids in Wyoming are 70 points ahead of kids in Mississippi. On WHAT? I cannot think of any common test kids in WY and MS take except NAEP and in 2007, Wyoming 4th graders scored 225 and Mississippi 4th graders scored 208. That’s 17 points, not 70, but I don’t think the President misread his teleprompter or suffers dyslexia. Whatever the differences, are they due, as the President claims, to the different standards in the two states? How about, differences in poverty. Thirty percent of the students in Wyoming are eligible for free or reduced price lunch. In Mississippi it’s 68 percent. My forthcoming book has a chapter "Poverty is Poison," a title I stole from Paul Krugman, with credit, and that, along with Dave Berliner’s new monograph on out-of-school factors in achievement should finally shut up the "poverty-is-no-excuse" crowd. (But they won’t).

Highlighted by cburell

What about those 13- and 14-year olds, only one third of whom "can read as well as they should?" This is NAEP data. This is one of the critics’ favorite statistics for stirring up anxiety. Roy Romer and Bob Wise use it all the time. About one third of these kids score at the NAEP "proficient" level. When the Center for American Progress, an organization that claims to be progressive, rolled out Leaders and Laggards, its joint venture with the Chamber of Commerce in 2007, CAP executive director, John Podesta declared that he found it "unconscionable" that no state had a majority of its students scoring at the NAEP proficient level. Unconscionable!

But, let’s ask, "If 13- and 14-year olds in other nations sat for America’s NAEP, how many of those countries would have a majority of students scoring at the proficient level?"

The answer, in reading at least is: Zero.

That’s what Richard Rothstein and colleagues found using a method first developed by Bob Linn. The highest scoring nation, Sweden, (Finland did not participate in this reading study) would have about one third of their students at or above the proficient level. The U. S., which scored very high, had 31%. Gary Phillips, former acting commissioner for statistics at NCES and now with the American Institutes for Research made similar estimates for math and science using TIMSS data. Only 5 of 45 nations would have a small majority of their students proficient in math and only two would clear that barrier in science. Further evidence, I would argue, that the NAEP achievement levels are set far too high.

Highlighted by cburell

The 'Fastest Growing Occupations'

What about those 30 fastest growing occupations? I’ve never seen that statistic presented in quite that way, but it also means that half of the 30 fastest growing jobs DON’T require a B.A. or better. But, the signal point about this statistic is that the 30 fastest growing jobs don’t account for many jobs. And the few that do are occupations like personal care aides, home health aides, nursing aides — low-paying service sector jobs needed in and for an aging nation.

Retail sales accounts for more jobs than the top ten fastest growing occupations combined. For every systems engineer needed by a computer firm, Wal-Mart needs about 15 people on the floor. The ten occupations accounting for the largest NUMBER of jobs in a Bureau of Labor Statistics projection from 2006 to 2016 were retail sales, cashiers, office clerks, registered nurses, janitors and cleaners, bookkeeping clerks, waiters and waitresses, food preparers and servers, customer service representatives, and truck and tractor drivers. I will show the falsity of Miller's and Duncan's linking of education and economic crises in detail later in the talk, but it terrifies me that our new President and Secretary of Education have apparently bought into the old falsehoods.

Little wonder that Diane Ravitch said that in education Obama was a third term for Bush and that Duncan was Margaret Spellings in drag.

Highlighted by cburell

The real causes of the current economic mess

I think one of the reasons we see so many such comments is captured a bit in the first two quotes. Neither Miller nor the President could bring himself to actually blame the schools for today’s economic catastrophe, but they laid on them some of the responsibility for any recovery. There is a long history of trying to link test scores to a nation’s economic health. That notion needs to become extinct

Let’s take a moment to reflect on the causes of the current mess. Banks used very little capital of their own to buy extremely risky real estate assets, granting subprime mortgages and mortgages on overpriced houses, often without even making credit checks. Then they used virtually unfathomable instruments such as credit default swaps to insure against loss. But insurance companies that insured those risks, like AIG didn’t have the capital to pay off the swaps when the banks’ bets went bad. The situation has produced a slight reworking of the opening rhetorical flourishes of that landmark document, "A Nation At Risk:"

We feel compelled to report to the American people that the business and financial foundations of our society are being eroded by a rising tide of mediocrity that threatens our very future as a Nation and a people. What was unimaginable a generation ago has begun to occur—companies that extolled themselves as models of excellent practices have deceived the American people with sloppy, undisciplined, and greedy practices that are driving Americans out of their homes, threatening their retirements, and dashing their hopes of a financially secure future. Indeed, if an unfriendly foreign power had attempted to impose on America the mediocre corporate financial performance that exists today, we might well have viewed it as an act of war.

Highlighted by cburell

Looking again at The Manufactured Crisis, I found all the myths that David [Berliner] and Bruce [Biddle] debunked [in their book The Manufactured Crisis] are still hanging around. They are just overlaid with another 15 years worth of data. I have a different perspective, plus the advantage of those 15 years of data. David and Bruce took each myth and demolished it individually. They organized their debunking around a collection of myths. I take a temporal approach. I organize around time and, while "A Nation At Risk," their signal event, was important, I think it all started earlier.

Highlighted by cburell

Bashing public schools in the USA since World War II

I see bashing schools as a longstanding predisposition that really took its current form just after World War II and I’d like to lay out today how that predisposition became an everyday habit. While Arthur Newman in his 1978 book, In Defense of the American Public School, has a section on “The always abundant criticism,” his reports are largely episodic into the Thirties and then he skips to the 70’s. In so doing, I think he misses the most important period of the rise of scapegoating.

Highlighted by cburell

most influential of all at the time, Arthur Bestor’s Educational Wastelands: The Retreat from Learning in Public Schools.

Note the use of the word “Retreat.” I think this is the first title ever that suggested that things had once upon a time been better, that there had been some golden age of American public education which we had somehow tarnished. In 1956, Bestor was interviewed by U. S. News & World Report. The interview ran under the headline, "We Are Less Educated than Fifty Years Ago."

Bestor made some remarkable historical errors, mostly by failing to take into account differences in the proportion of students finishing high school in 1906 vs. 1956 and changes in the socioeconomic status of those getting a diploma over that 50-year period. I say remarkable because Bestor was a historian and you’d think a historian would have noticed such things.

Highlighted by cburell

The schools never recovered from the criticism unleashed by Sputnik, and quickly became society’s scapegoat of choice for perceived social crises. Since Sputnik, the driving force behind school reform efforts has been fear of the future with fear often serving as a means to gain power and control over the schools.

Highlighted by cburell

When America’s cities went up in flames in the 1960’s schools once again were blamed. This time, at least one person saw the scam for what it was. Fred Hechinger, writing in the New York Times in 1967 observed

Almost 10 years ago, when the first Soviet Sputnik went into orbit, the schools were blamed for America’s lag in space. Last week, in the Senate, the schools were blamed for the ghetto riots.

In each case, the politicians’ motives were suspect. Their reflex reaction, when faced with a national crisis, is to assign guilt to persons with the least power to hit back. The schools, which are nonpolitical but dependent on political purse strings, fill the bill of emergency whipping boy.

Highlighted by cburell

Earlier critics did not have the hammer of tests to pound schools with. NAEP did not exist until the 1970’s and it remained largely invisible until the 1990’s. SAT scores were stable up to 1963. Most states did not have state testing programs until the 1970’s. All Norm-Referenced tests except the various Iowa’s had floating standards so there was no way to tell if the 50th percentile in 1960 represented the same level of achievement as in 1950. International comparisons began in the 1960’s but the early ones contained too many methodological flaws to make them credible.

Highlighted by cburell

Testing was now firmly cemented in place as THE instrument for measuring kids, teachers, and schools. A report to add steel reinforcement to that notion would come along soon. President Reagan’s Secretary of Education, Terrell Bell, hoped for what he called a "Sputnik-like event" to shock Americans about the terrible state of their schools. Lacking such an event, Bell in 1981 reluctantly formed the National Commission on Excellence in Education. One of the driving forces of that commission, Harvard Physicist, Gerald Holton, at first refused the invitation to serve because Reagan was moving to dismantle some science and education programs that Holton supported and he was surprised and dismayed by the makeup of the commission—not a single national expert in education. It was an omen—the control of education would, over the next 20 years be stripped from educators.

But Holton was coaxed in and "Reagan gave us our marching orders: Bring God back into the classroom. Encourage tuition tax credits for families using private schools. Support vouchers. And please abolish that abomination, the Department of Education. As we left, I detected no visible dismay in our group. I wondered if we were all equally stunned."

The report, "A Nation At Risk," did not mention any of Reagan’s "marching orders" and that caused a schism in White House. Adviser Ed Meese and other conservatives implored Reagan to reject it because it ignored Reagan’s marching orders. Moderates Jim Baker and Mike Deaver urged acceptance because it contained many issues Republicans, including Reagan, could campaign on.

Highlighted by cburell

After the Cold Warrior rhetoric came 13 indicators of the risk. All reflect test scores. All accentuate the negative, even when they don’t have to. One of the indicators could have said "Scores on the Iowa Tests of Basic Skills and Iowa Tests of Educational Development have risen for seven consecutive years," but that information was not among the 13 pieces of data.

Highlighted by cburell

Some of the indicators were highly selective. For example, one says, "There was a steady decline in science achievement scores of U. S. 17-year-olds as measured by national assessments." If you look at NAEP trends, you will indeed find such a trendline for 17-year-olds. But not for 13-year-olds or 9-year-olds, the other two ages assessed. And the trends in reading and math show no hint of a decline at any age. Thus the commission had 9 trendlines, only one of which could be used to support crisis rhetoric and that was the one it used.

Highlighted by cburell

"A Nation At Risk" brought together two assertions that have become gospel: educational quality can be measured by standardized tests and we must raise our test scores to compete in the global economy. High test scores = healthy economic nation. Those are the leitmotifs of the 26 years since the report appeared. You can hear them reflected over and over again in the quotes from President Obama and secretary Duncan that opened this talk, but they took their current form in 1983.

Highlighted by cburell

Bell dispatched a team of educators headed by assistant secretary Chester Finn to Japan to see if we could import their education system. One of the team members, Herb Walberg, declared to the Washington Post, "I think it’s portable. Gumption and willpower, that’s the key." Otherwise Japan would eat our lunch. In the article, there is no indication that people perceived Walberg’s assertions as absurd.

You remember Japan. Its economy was a miracle. Its students aced the tests in international comparisons. Then, about 1990, the Japanese discovered that the Emperor’s Palace and Grounds really weren’t worth more than the entire state of California, and its economy sank into the Pacific, soon taking the other Asian Tiger nations with it. The Japanese now speak of the 1990’s as the lost decade. They’ve never actually had a particularly good year since 2000 and they’re back in a heap of economic hurt: They officially declared themselves in recession again in 2007 and in 2008, the Japanese economy shrank three times as much as the U. S. economy. Their kids still ace tests.

Highlighted by cburell

In 1987, reviewing a proposal to revamp NAEP, Bob Glaser, writing for the National Academy of Education said,

Most of those personal qualities that we hold dear—resilience and courage in the face of stress, a sense of craft in our work, a commitment to justice and caring in our social relationships, a dedication to advancing the public good in communal life—are exceedingly difficult to assess. And so, unfortunately, we are apt to measure what we can, and eventually come to value what is measured over what is left unmeasured. The shift is subtle and occurs gradually.

Highlighted by cburell

Well, maybe there’s hope.

One way of chasing out the fear mongers, maybe, is to show them that there’s so much more to education than any test can measure. The Sunday, April 5, 2009 New York Times carried a page 1 article about activities that the Scarsdale district is using teach empathy. The hook for the article is probably that it’s in Scarsdale, but it also reports on major efforts in Los Angeles and 18 states. "Empathy lessons are spreading everywhere amid concerns over the pressure on students from high-stakes tests and a race to college that starts in kindergarten."

Highlighted by cburell

The Educational Testing Service is adding a "Personal Potential Index" to the Graduate Record Examination with ratings by instructors on “knowledge and creativity; communication skills; capacity for teamwork; resilience; planning and organizational skills; and ethics and integrity.” ETS says that students need these qualities to succeed in graduate school. Why limit them to graduate school? Somebody should tell Arne about ETS’ plans.

Highlighted by cburell

From my experiences living in East and Southeast Asia and from what I’ve read, I cannot imagine students in any of those nations asking questions, or any employee suggesting to a supervisor that there’s a better way of doing something.

Highlighted by cburell

People, though, especially the media and those who would like to take advantage of one of the largest untapped markets anywhere—the public schools—still make the connection between test scores and the economy.

Highlighted by cburell

The schools cannot buy a good headline. The Progress in International Reading Study gets very few headlines. That might be because the US does well in that international comparison. In 2001, three countries scored higher, 9 were not significantly different, and 23 were lower. In 2006, with 45 countries, 10 scored higher, 13 the same and 30 lower. The U. S. score was 542 in 2001 and 540 in 2006. That difference is the same as a two-point difference on the SAT and well above the international average of 500. But the headline in Education Week, the only outlet I know of that even covered the results at all, was "America Idles on International Reading Test?" Why not "America Continues To Do Well On International Reading Test?" In any case, if America idles, the rest of the world is in neutral, too. Only 10 of the 28 countries that were in both assessments changed more than 10 points in either direction.

Highlighted by cburell

One way of countering the fear mongers is to determine whether or not they are using the right statistic or whether there are statistics that are more meaningful. The international comparisons, TIMSS, PISA, PIRLS, all come to us mostly in terms of average scores and how those scores affect a country’s ranking among other countries in the world. But as Hal Salzman the Urban Institute and Lindsay Lowell of Georgetown pointed out in a paper in Nature last May, the average scores don’t mean anything. They write "If as we argue, average test scores are mostly irrelevant to as a measure of economic potential, other indicators do matter. To produce leading-edge technology, one could argue that it is the numbers of high-performing students that is most important in the global economy. These are students who can enter the science and engineering workforce or are likely to innovate whatever their field of study. Remarkable, but little noted, is the fact that the United States produces the lion’s share of the world’s best students."

I would temper that by pointing out that they define "world’s best" terms of test scores, but that’s the currency of the day and in that currency no one else even comes close. It is very difficult to attain Level 6 on the PISA tests, the highest on the scale. New Zealand tops the world with 4% and Finland is second with 3.9%. But that 3.9% for Finland only translates into 2000 warm bodies. The U. S. has 67,000. Japan is second with about 34,000. The UK has 21,000.

Highlighted by cburell

The bad news is that we have the most lowest-performing students aside from Mexico. So there’s plenty of work to be done on the system, but we don’t have to say, as Duncan just did and as hundreds have across the years, we have a "crisis" in education.

Highlighted by cburell

If test scores were so important to global competitiveness, you’d think our ranks on global competitiveness would be comparable to our ranks in the international test comparisons. But they’re not. There are two organizations that make it their business to rank nations on global competitive, the Institute for Management Development in Lausanne, Switzerland and the better known World Economic Forum which had its annual do recent in Davos, Switzerland. Now it is really going to be interesting to see what the current debacle does to the rankings, but as of 2008-2009 reports, the U. S. was number one.

The IMD ranks 55 nations and the U. S. took over from Japan in 1994.

Highlighted by cburell

The IMD wonders how much longer the U. S. can keep the lead. It notes that Singapore has been rising fast. But who cares? As Sam Dillon of the New York Times observed recently, comparing a diverse nation of 300 million with tiny city states such as Hong Kong and Singapore makes little sense. Which reminds me, seldom do the fear mongers take the cultural context of schools into account when discussing test scores. They seldom mention that everyday thousands of poor Malaysians cross into Singapore to do the dirty work. The long-term “guest workers,” mostly Filipinos, are forbidden to bring their families. These factors save Singapore a lot of work in educating children of the poor.

Highlighted by cburell

What is important about the IMD rankings and even more so the WEF rankings is that they reveal how complex the idea of competitiveness is. Education is only one aspect of those rankings.

The WEF has established what it calls "The Twelve Pillars of Competitiveness." I don’t have time to go into detail, but one way of countering the fear mongers is to try to make them aware of this complexity.

First comes institutions which has to do with bureaucracy, corruption, transparency, trustworthiness, accountability, etc. As I say, it will be interesting to see what this year’s report looks like.

The second pillar is infrastructure. We hear a lot about that these days, but before Katrina blew into town and that bridge collapsed into the Mississippi, I imagine few Americans thought much about infrastructure. Especially in terms of competitiveness. We took our roads, rails, ports, and airports for granted.

Third is Macroeconomy. This is where the US does worst because the WEF hates deficits—that means money that could be used to increase productivity has to be used as interest on loans.

I’ll just name the others, health and primary education, higher education and training, goods market efficiency, labor market efficiency, financial market sophistication (our financial markets sophisticated themselves right out of business), technological readiness, market size, business sophistication and, most important, innovation. Innovation, where the U. S. is #1, is the most important because it is the only pillar that does not at some point yield diminishing returns. You can only gain so much by, say, making planes bigger and faster, but innovation has no limit. I this connection, I would point out the Bob Sternberg, Dean of Arts and Sciences at Tufts has observed that our obsession with testing has given us our most successful tool for stamping out creativity.

Highlighted by cburell

The WEF ranks 134 nations and the U. S. has been #1 for most of the last decade, occasionally slipping to #2. If the forecasts of A Nation At Risk had been correct and if, as the critics claim, the schools never improved, how then did we get ranked #1 in global competitiveness and enjoy the longest sustained expansion in the world, an expansion that was real—productivity soared in the 1990’s.

Highlighted by cburell

Finally, let me return to NAEP, but rather than inundating you with numbers of numbers, I’ll just say this: Because of the changing ethnic makeup of this nation, and because of the fact that black and Hispanic students score still lower than whites, analyzing NAEP data simply at the national level makes no sense. In fact, it makes you the victim of Simpson’s Paradox. Simpson’s paradox shows up whenever the whole group analyzed shows one pattern but the subgroups show a different pattern.

In 2002, the College Board lamented that there had been no change in the SAT verbal score since 1982. This was true for the national sample. But when I analyzed the trends by ethnicity, I found gains for all groups, some of them quite large. The reason for the apparent paradox was simply the changing demographics. In 1981, whites made up 85% of all SAT testtakers, in 2005, when I conducted the analysis the figure was 63%. The minorities were improving. But because the scores were still lower than whites and because minorities were making up an ever increasing share of the total pool, their lower scores attenuated the overall average.

Highlighted by cburell

The same thing is true of NAEP. While there have been some areas of gain overall, if you look at the NAEP trends by ethnicity, the only place you see stability is in 17-year-old white kids. Everybody else is up and in groups up dramatically. For example, in mathematics, black 9-year-olds gained 34 points between 1978 and 2004, black 13-year-olds gained 34 points and black 17-year-olds gained 15 points.

Highlighted by cburell

And just as you must cannot speak of "American" scores without looking at ethnic breakouts, neither can you speak of American schools without looking at breakouts by poverty. Yes, other countries have poor kids but many fewer. In one UNESCO study, the U. S. was 20th among 20 developed nations. Here’s an example of what poverty does to test scores taken from PIRLS.

Top country Sweden, score 561

U. S. overall 542

International Average 500

The 14.5% of American students in schools with less than 10% of their students in poverty 589

The 19.5% of students in schools with 10% to 25% in poverty, 567. The 29.8% of students in school with 25-50 in poverty 551—if this group constituted a nation it would rank 4th among the 35 countries.

That’s 64% of American students scoring at the top.

The 21.3% of American students in schools with 50-75% in poverty, 519, still above the international average.

Only the 15.1% of students in schools with more than 75% of their students in poverty score below the international average at 485.

Highlighted by cburell

I fear that there is no way to counter the fear mongers directly. They have an agenda most of which revolves around controlling the curriculum and instruction and/or replacing the public schools with a private system.

They are subject to shame, however. About 10 years ago Alex Molnar put together a loose confederacy of people to act as an underfunded countweight to the Heritage Foundation, Hoover Institution, Manhattan Institute, etc. At the first meeting I looked around at the 25 or so people in the room and said "This will never work." The reason, I thought, was that the people in the room already had very full lives without taking on additional efforts on behalf of the schools. But, ten years later, now co-headquartered at the University of Colorado and Arizona State, there are about 120 "fellows" as we are called engaged in a number of endeavors. Among the more successful, I think, is the Think Twice project known internally as the Think Tank Review Project. The Heritages and Hoovers of the world along with some professors like Paul Peterson at Harvard and Jay Greene at Arkansas had taken to publishing un-peer reviewed papers which, on close examination, presented selective data leading to pretermined conclusion. The Think Twice project hires scholars to review these advocacy research papers as one would a manuscript submitted to Ed Researcher or AERJ and publishes the results. The originals are credited for sound conclusions, faulted for overreaching, which is a common theme in the papers.

Highlighted by cburell

It is said that if you tell a lie often enough people will believe. One can only hope that the same applies to the truth.

Highlighted by cburell