America’s Mediocre Test Scores

At a time when the national conversation is focused on lagging upward mobility, it is no surprise that many educators point to poverty as the explanation for mediocre test scores among U.S. students compared to those of students in other countries. If American teachers in struggling U.S. schools taught in Finland, says Finnish educator Pasi Sahlberg, they would flourish, in part, because of “support from homes unchallenged by poverty.” Michael Rebell and Jessica Wolff at Columbia University’s Teachers College argue that middling test scores reflect a “poverty crisis” in the United States, not an “education crisis.” Adding union muscle to the argument, American Federation of Teachers president Randi Weingarten calls poverty “the elephant in the room” that accounts for poor student performance.

But does the room actually contain the elephant?

To prove that poverty is the major factor driving America’s meager academic achievement, at least two of the following three claims need to be established:

1. Poverty is related to lower levels of student learning.

2. America’s poor students perform worse than other countries’ poor students.

3. The poverty rate in the United States is substantially higher than the rates in countries with which it is compared.

Let’s examine each in turn.

Is Poverty Related to Lackluster Learning?

To this first question, the answer is obviously in the affirmative. That’s not to say “poor children can’t learn.” It is to say, rather, that there’s long been a clear connection between families’ socioeconomic status and students’ academic achievement. As can be seen in Figure 1a, states with higher percentages of students from low-income families report lower average scale scores in 8th-grade math on the National Assessment of Educational Progress (NAEP). The same connection between poverty and academic performance can be observed at the school level (see Figure 1b).

Why do kids from low-income families tend to score so much lower on average than their more-affluent peers? Is it something about poverty itself, that is, a lack of financial resources in the family? This is likely the case, as financial stress can create “toxic” conditions in the home and also make it difficult (if not impossible) for parents to afford the tutoring, educational games, summer camps, afterschool activities, and other educational experiences that middle-class and upper-middle-class students take for granted (and that almost surely boost their achievement).

But it’s not just about money. Poverty is associated with a host of other social ills that have a negative impact on learning. For instance, children in poverty are much more likely to be living in single-parent families headed by young, poorly educated mothers. Poverty is also associated with higher rates of alcoholism and other substance abuse in the home; greater incidence of child abuse and neglect; and heightened family involvement in the criminal justice system. All of these are well-known “risk factors” that are associated with lower test scores as well as with a greater likelihood of dropping out of high school.

So, yes, in general, poverty and factors correlated with low family income are strongly related to low test scores.

Do U.S. Students from Low-Income Families Underperform Their Peers Overseas?

The next question is whether U.S. students from low-income families are lower-scoring than those in other countries. To explore this question, we’re obliged to wrestle with measurement issues. The problem is complicated because no international data set contains both good measures of family income and good measures of student test-score performance.

The best available information is to be found in the data collected by the Program for International Student Assessment (PISA), which is sponsored by the Organization for Economic Co-Operation and Development (OECD). PISA, for its own analyses, uses an index of economic, social, and cultural status (ESCS) that looks at parent occupation and education, family wealth, home educational resources, and family possessions related to “classical” culture. PISA analysts use the index to stratify each country’s student population into quartiles.

Not everyone will agree with the way the ESCS index is constructed, but the data presented in Figure 2 are nonetheless quite instructive. The test scores of students in the bottom quartile of the ESCS index are plotted against those of students in the top quartile. If students in these two quartiles did equally well in each country (as compared to similarly situated students in other countries), then the dotted regression line displayed in green would have a steeper slope, and every dot would fall exactly on that line. As you can see, the actual pattern is not that perfect, as some countries, such as Belgium and France, are relatively better at teaching the higher-status students, while other countries, such as Canada and Finland, do relatively well at instructing students from lower-status families. But notice that the United States falls almost exactly on the regression line. It does equally well (or equally poorly, if you prefer) at teaching its least well-off as those coming from families in the top quartile of the ESCS index.

If we look at a different marker of socioeconomic status, parental education levels, we find a similar pattern. In the U.S., for instance, parents without a high school diploma are much more likely to be in poverty than their better-educated peers, and their children are much more likely than their peers to be low-performing and to drop out of school themselves.

In a study that examined whether some countries are particularly effective at teaching students from disadvantaged backgrounds, Eric A. Hanushek, Paul E. Peterson, and Ludger Woessmann find little difference in the rank order of countries by the performance of students from families where a parent had a college education and the rank order of countries by the performance of students whose parents had no more than a high school diploma. They find that if a country is comparatively effective at teaching the first group, it tends to be no less effective (as compared to others) at teaching the second. The United States performs as expected, proving not to be especially effective at teaching students from the best-educated or the least-educated families. The authors write,

Overall, the U.S. proficiency rate in math places the country at the 27th rank among the 34 OECD countries that participated in the Program for International Student Assessment (PISA). That ranking is somewhat lower for students from advantaged backgrounds (28th) than for those from disadvantaged ones (20th).

There is no evidence that disadvantaged students in the United States are underperforming other countries’ disadvantaged students. If anything, it is the “advantaged” U.S. students (those whose parents have a high level of education) who are falling short in international comparisons.

Is America’s Child-Poverty High Compared 
to Rates Elsewhere?

So far we’ve acknowledged that poverty is, in fact, strongly (and negatively) related to achievement. But we’ve also demonstrated that disadvantaged students in the United States are performing as expected, given the performance of better-situated U.S. students.

But if more students are poor in the U.S. than in other countries, it is still possible that students from low-income families are dragging down U.S. national averages. If that is true, poverty could still be the elephant in the classroom.

But does the U.S. have a greater proportion of low-income students than other countries?

For those educators quoted at the beginning of this essay, the answer is yes. They assert that the U.S. has a sky-high child-poverty rate compared to other developed countries.

To support their claim, they use a measure that assumes all families with less than half the median income in the country are by definition “poor.” Figure 3 shows relative child-poverty rates for selected countries.

In the U.S., median family income is about $52,000 per year, so any family earning less than $26,000 a year is said to be poor. The measure excludes any income from governmental transfers.

Relying on measures of relative poverty is appealing for its simplicity, but it is a highly misleading approach because it’s more a measure of income inequality than of poverty.

To see how relative poverty rates can mislead, let’s look at how they compare to absolute poverty rates for the general population in the American states. In Figure 4, we report the proportion of people living in households that earn less than half of their own state’s median income (basing state median incomes on the 2013 Census Current Population Survey). We also show each state’s absolute poverty rate as it is traditionally defined: the percentage of all people in the 
state living in households below the federal poverty line, which is currently set at $24,250 for a family of four.

For some states, whether one looks at relative poverty or at absolute poverty makes little difference. Arizona, Mississippi, and Louisiana have a lot of poor people however you slice the data.

But notice where wealthier states like Massachusetts and Connecticut appear on the graph. Their absolute poverty rates are among the lowest in the country. But their relative poverty rates are above average—higher than Texas, Tennessee, and Oklahoma. Massachusetts has a higher relative poverty rate than Georgia, Kentucky, and Alabama.

Of course, Massachusetts doesn’t really have more poverty than Alabama—but it does have more income inequality.

The same dynamic plays out when we use relative poverty rates to compare countries. Many of the U.S. households that are counted as poor on a relative measure would be considered middle class on an absolute measure.

Using 2010 data, Timothy Smeeding, founder of the Cross-National Data Center in Luxembourg, reports absolute poverty using a methodology that takes into account all forms of income, including social welfare benefits (see Figure 3). By this measure, the U.S. absolute poverty rate is lower than the United Kingdom’s, virtually the same as Germany’s, and just barely higher than Finland’s. To be sure, the U.S. still has too much poverty. But once social welfare benefits are included, and we look at absolute instead of relative poverty, the U.S. is hardly an outlier.

It’s important to note that the absolute poverty rates shown in Figure 3 are for the general population, not for children. It’s possible that absolute child-poverty rates would look quite different. But we have no way of knowing, because the data to calculate those rates across a large number of countries do not currently exist.
What we can say definitively is that relative poverty rates can be highly misleading. We ran a regression analysis to estimate the relationship between states’ absolute and relative poverty levels and student achievement, and the result was clear: absolute poverty is a powerful predictor of achievement, while the relationship between relative poverty and test scores in the U.S. is weak and not statistically significant (see Figure 5).

Relative poverty is also a weak predictor of student achievement internationally. In another analysis, we compared relative child-poverty rates  to PISA mean math scores in 2009—and once again found only a weak and statistically insignificant relationship.

In short, relative poverty rates, which are only weakly related to student achievement both in the U.S. and abroad, are erroneously used to explain America’s academic struggles. They seem to indicate that the U.S. has an unusually large population of low-income individuals, but in fact they simply demonstrate an unusually high degree of income inequality. Using absolute poverty rates—which are related to student achievement within the U.S.—we see that the proportion of Americans who are poor is quite typical by international standards.

Conclusion

Critics of education reform are certainly correct when they say that poverty is a major factor in lackluster academic performance.

Still, poverty is an issue for virtually every nation on the planet. Where reform critics get it wrong is when they claim that America’s average scores are dragged down by the particularly poor performance of low-income students, or that the advantaged kids are doing just fine.That is objectively untrue. And its scores are not dragged down by an unusually high proportion of poor students, as measures of absolute poverty find the U.S. not to be an outlier at all.

America’s mediocre performance is remarkably consistent. Yes, affluent students outperform poor students. But they don’t outperform their peers overseas.

This doesn’t imply that reform, as currently formulated, is on the right track. Why U.S. student performance is mediocre is a topic worthy of study and debate, as is how to help students at all points on the economic spectrum perform better.

What it does show is that poverty can’t explain away America’s lackluster academic performance. That excuse, however soothing it may be to educators, politicians, and social critics, turns out to be a crutch that’s unfounded in evidence. We need to stop using it and start getting serious about improving the achievement of all the nation’s students.

Michael J. Petrilli is president of the Thomas B. Fordham Institute, where Brandon L. Wright is managing editor and policy associate.

---

Update, November 18, 2015:

The method used to calculate absolute poverty rates in 2010, as reported in Figure 3 of “America’s Mediocre Test Scores,” required estimations from data made available by Timothy Smeeding. The note to Figure 3 does not include the estimating procedures used to calculate absolute poverty rates for 2010, which are availablehere. Some commentators have questioned those procedures, but similar results are obtained for 1999-2000 by Smeeding when poverty rates are calculated as 125% of the U.S. Department of Agriculture’s poverty line in his 2006 paper, “Poor People in Rich Nations: The United States in Comparative Perspective.” In both cases U.S. poverty rates are only modestly above that in several comparable countries. Differences are unable to explain the much larger differences in student achievement between the United States and other countries. For further discussion, please see “Is America’s Poverty Rate Exceptional? It depends on how you define poverty.”

	Percentages as given in Figure 3, “America’s Mediocre Test Scores,” Petrilli and Wright, 2015	Percentages as given in “Poor People in Rich Nations: The United States in Comparative Perspective,” Timothy Smeeding, 2006 (Using data from 1999-2000 and calculating poverty as 125% of U.S. poverty line)
United Kingdom	18.8	32.8
United States	15.3	19.5
Canada	9.9	17.5
Finland	14.6	17.5
Germany	15.2	17.3
Netherlands	7.4	17.3
Austria	*	15.3
Sweden	*	13.8
Belgium	*	12.2

* Not mentioned in Figure 3, “America’s Mediocre Test Scores,” Petrilli and Wright, 2015