BOTTOM LINE: Correlation between State-level SAT average scores and test scores of representative (the average) students is almost ZERO. Knowing the state SAT average tells you virtually NOTHING about the quality of state education.
An old saying among South Carolina educators – thank heavens for Mississippi. No matter the issue — test scores, teacher salaries, class size — Mississippi was always worst.
But the 2011 combined SAT scores (created by adding the math-reading-writing average for each state) show Mississippi at 1,660 points, South Carolina at 1,436 points and second from the bottom! Once again, a big fail!?? Below is a sample of eleven states. See the problem! (Click here for the original state-by-state data. This article was written using 2011 data.)
| State | Combined SAT Index |
| Illinois | 1,807 |
| Minnesota | 1,778 |
| Iowa | 1,777 |
| Kentucky | 1,711 |
| Tennessee | 1,710 |
| Mississippi | 1,660 |
| New Hampshire | 1,559 |
| Massachusetts | 1,549 |
| North Carolina | 1,475 |
| Florida | 1,447 |
| Georgia | 1,445 |
| South Carolina | 1,436 |
| Maine | 1,391 |
How can we account for these differences? Should we send a swat team to Mississippi to see what they are doing to have such high scores? How does South Carolina always come out near the bottom?
But now see the same table of eleven states with an added column – the percentage of graduating high school seniors who took the test. In all but a few states, taking the SAT is voluntary. If your local colleges and universities don’t use the SAT results, why bother – unless you are well-to-do and applying to a selective school in the Ivy’s, on the West Coast, or such. The percentage of graduating seniors taking the test varies from a low of 3% in North Dakota, 4% or so in South Dakota, Iowa and Mississippi, to almost 100% in Maine, Connecticut, and Massachusetts.
| State | % Srs. taking | SAT Index 2011 |
| Illinois |
6% |
1,807 |
| Minnesota |
8% |
1,778 |
| Iowa |
4% |
1,777 |
| Kentucky |
6% |
1,711 |
| Tennessee |
11% |
1,710 |
| Mississippi |
4% |
1,660 |
| New Hampshire |
85% |
1,559 |
| Massachusetts |
97% |
1,549 |
| Florida |
75% |
1,447 |
| Georgia |
81% |
1,445 |
| South Carolina |
71% |
1,436 |
| Maine |
98% |
1,391 |
The new column gives an estimate of the percentage of graduating Seniors who took the SAT. Mississippi had only 4% of their Seniors taking the exam, while South Carolina had 71%. If you look at this subset of states, it appears that the best way to have a high average for your state is to have a smaller percentage of your students actually taking the test.
So Mississippi apparently played 4% of “varsity” scholars against our 70% or so range of more average students. Are we surprised that the average varsity scholar in Mississippi does better on standardized tests than the average South Carolina student? We prevent that from happening in high school sports by putting similar schools in similar divisions, and having most schools actually play – but remember for many states taking the SAT is voluntary and the percentage taking them is extremely uneven, ranging from 3% to almost 100%.
The takeaway — SAT State Average scores are WORTHLESS for comparing states. The typical test taker in some states can be far from average, part of the scholastic elite, or in other states, which require most students to take the SAT, truly average.
If someone tells you that South Carolina has a bad educational system AS DEMONSTRATED BY SAT scores, just remind them of how good Mississippi is! And know that the correlation between these average SAT scores and a true measure of academic performance on the “Nation’s Report Card” is zero. Knowing the state SAT average score tells you NOTHING about the performance on standardized tests taken by a representative sample of students in that state
So do we have a better way to compare the academic performance of states if the SAT comes up so short, because who takes it varies so much from state to state?
FAIR STATE COMPARISONS
To actually compare states fairly we need a representative sample of students in each state taking the same test. The SAT does not meet that standard, but there is a common test given by the U.S. Department of Education that does exactly that. A representative sample of fourth graders and eighth graders in EVERY state takes the same exam most years as part of The Nations Report Card. Check out the table below that the same eleven states in our SAT table above. These scores are taken from the 2010 National Assessment of Educational Progress (NAEP) scores. I simply added the 4th grade reading and math scores to the 8th grade reading and math scores to create a simple index to compare states.
NAEP Index for the 11 states
in SAT order
| State |
NAEP index |
| Illinois |
1,007 |
| Minnesota |
1,037 |
| Iowa |
1,013 |
| Kentucky |
1,017 |
| Tennessee |
981 |
| Mississippi |
962 |
| New Hampshire |
1,046 |
| Massachusetts |
1,065 |
| Florida |
1,004 |
| Georgia |
1,000 |
| South Carolina |
994 |
| Maine |
1,025 |
The SAT scores unfairly put Illinois on the top and Maine on the bottom of this short list. Notice now that Illinois is only slightly above average, Massachusetts way up, and South Carolina is no longer second from the bottom: in this short table Tennessee is now second from the bottom, and in the table for all 50 states thirteen states have a lower NAEP index score than does South Carolina. See the table below for these same eleven states sorted by this NAEP index.
NAEP Index for the 11 states
in NAEP order
| State | NAEP index |
| Massachusetts |
1,065 |
| New Hampshire |
1,046 |
| Minnesota |
1,037 |
| Maine |
1,025 |
| Kentucky |
1,017 |
| Iowa |
1,013 |
| Illinois |
1,007 |
| Florida |
1,004 |
| Georgia |
1,000 |
| South Carolina |
994 |
| Tennessee |
981 |
| Mississippi |
962 |
Not only is South Carolina no longer at the bottom – for all fifty states 13 have lower NAEP index scores than does South Carolina and another 3 states are only slightly higher. So South Carolina is essentially tied for 35th in the nation (rather than 49th )..
One factor we know greatly influences a state’s performance on standardized tests is social class or income level. A smaller percentage of children in poverty typically lead to higher test scores. The table below shows the same eleven states with scores arranged by the NAEP index, and a new column showing an estimate of the number of children living in poverty using data from the US Census Bureau. Note that this measure is only for children at and below the official poverty line, and does not include reduced lunch children who are slightly above the official family poverty line but still qualify for lunch aid and often for Medicaid insurance. These then are the poorest of the poor, and not also the near poor often included in the “free and reduced lunch” measures used in South Carolina.
Our 11 sample states with scores arranged by the NAEP Index,
with a column showing the Poverty Index (% children below poverty line)
State |
NAEP |
Poverty Index |
| Massachusetts |
1,065 |
14% |
| New Hampshire |
1,046 |
10% |
| Minnesota |
1,037 |
15% |
| Maine |
1,025 |
18% |
| Kentucky |
1,017 |
26% |
| Iowa |
1,013 |
16% |
| Illinois |
1,007 |
19% |
| Florida |
1,004 |
23% |
| Georgia |
1,000 |
25% |
| South Carolina |
994 |
26% |
| Tennessee |
981 |
26% |
| Mississippi |
962 |
33% |
Notice that as the poverty measure increases, the NAEP index decreases. The average test score in poorer states is lower than in more well-to-do states. (Though not covered here, this same impact can be seen between nations. Since the U.S. has considerably more childhood poverty than other well-to-do countries, we often have lower average test results.)
But the small table I used for illustration left out 39 states. The table below gives the NAEP index score and poverty rate for ALL thirteen states with a lower NAEP index than South Carolina.
States below South Carolina in the NAEP 2011 Combined Index
with information on the % under 19 in poverty
| State |
Comb 4th-8th grade Reading Math NAEP |
% under 19 in Poverty |
| South Carolina |
994 |
26% |
| Arkansas |
992 |
28% |
| Oklahoma |
992 |
25% |
| Alaska |
989 |
13% |
| Hawaii |
988 |
14% |
| Arizona |
987 |
24% |
| Nevada |
986 |
22% |
| Tennessee |
981 |
26% |
| Alabama |
979 |
28% |
| West Virginia |
978 |
25% |
| California |
973 |
22% |
| New Mexico |
971 |
30% |
| Louisiana |
969 |
27% |
| Mississippi |
962 |
33% |
Surprised? On a test that measures a representative sample of students, South Carolina is not second from the bottom, but actually does better than many states with a lower poverty rate (and not just Mississippi, which is the poorest state by this poverty index measure). It is simply not true that South Carolina ranks near the bottom of all states in educational achievement, despite the rhetoric that suggests our low SAT scores show the state schools do a poor job. If we want higher SAT average scores, we might discourage local colleges and universities from using the SAT for admission so only kids going out of state would bother to take the test – we could aspire to get the percentage taking the SAT test down to under 10% and have as high an SAT score as Mississippi does.
Technical detail – fuller data for all 50 states, correlations and regressions
At the end of this section is a table with the data used for all 50 states. To summarize the relationships, analysts often use simple descriptive statistics.
It turns out that there is a very close relationship between the percentage of High School graduates taking the SAT test and the average state-level SAT score. The chart below shows a scatterplot for each state, with each blue point showing the percentage taking the SAT on the vertical axis, and the same state’s average SAT score on the horizontal axis.
CHART 1: For each state, the Percentage taking the SAT (vertical axis)
by the Mean Composite SAT score (horizontal axis)
I labeled several of the points by the state name. All the labels in black (like HI, NV, CA, AK, MS and TN) are states that have a lower NAEP index score than South Carolina. Notice that many of them have a much higher SAT score than the SC 1,436 (like Arkansas with 1,692, Oklahoma with 1,683, and Mississippi with 1,660). Other state, like Maine, New Jersey, Connecticut, Massachusetts, and New Hampshire, had lower SAT scores than these but much higher NAEP index scores. Especially noteworthy, Maine had the lowest SAT average in the country at 1,391, but on the NAEP index finish in the top third of states
Looking at this chart should convince one that there is a strong relationship between the state SAT scores and the percentage of Seniors taking the SAT. We can use descriptive statistics to formalize the strength of that relationship.
The solid line in Chart 1 is the best-fitting least-squares line, also known as the linear regression line. The equation for that regression line is:
Predicted SAT = 1715 – 2.9 * % taking SAT
Some examples:
If 98% take the SAT, the predicted SAT score is 17.15 – (2.9 * 98%) = 1431
Actual states near this percentage, ME at 1,391, CT at 1,535 and MA at 1,549.
For 50% taking, the predicted SAT score is 1570 (a lower percentage predicts a higher SAT)
Actual states near this percentage, NV at 1,460, OR at 1,540 and AK at 1,513
For 3% taking the predicted SAT score is 1706.
Actual states near this percentage, SD at 1,737, MS at 1,660, and ND at 1,759
The Pearson correlation R = 0.87, the R-squared (variance explained) is 0.76
Check the table and graph to see how close the predictions come to the outcome. So once one knows the percentage of seniors taking the SAT, there is not much left to explain. Just that fact alone will give you a good estimate of the SAT result, even for a very poor state like Mississippi. And knowing this, you are not likely to be taken in by those who claim the South Carolina SAT results show something significant about how South Carolina compares educationally to other states, except that we sure have a lot of kids taking the SAT, and the Princeton Testing Service who charges for the test sure must love South Carolina (and not Mississippi).
The second chart shows the scattergram for all 50 states for the NAEP index vs. the children’s poverty rate.
In the formal statistical analysis of the second chart, the formula for the linear line is:
Predicted NAEP index = 1075 – 3.22 * % Poverty
R= 0.73 and R-squared = 0.53. So over 50% of the variance can be explained just by the poverty rate.
This correlation is lower than the one for SAT vs. percentage taking. For example, in the lower left portion of the chart you see Hawaii and Alaska, with relatively lower poverty rates, are doing less well on their average than is South Carolina, while others (like MA, the highest on the Y-axis) do better than expected. Figuring out why would be interesting. While it might be interactions with ethnic identities that have little to do with actual school programs, some of the remaining differences might be explained by differences in actual school organization and instruction methods. Looking for “outliers,” for exceptions to the general rule and understanding why these are exceptional can advance the state of understanding.
Some example calculations and states close to the predicted results:
At 10% poverty, the Predicted NAEP = 1075. States close to this poverty level and their actual score — NH 1046, MA 1065, CT 1032
At 21% poverty, the predicted NAEP = 1007. States close to this poverty level and their actual NAEP index scores — MO 1010, NY 1006, IN 1014
At 33% poverty, the Predicted NAEP = 969. States close to this poverty level and their actual NAEP index scores — MS 962, NM 971, Alabama 979
The table below shows the data for all 50 states, sorted by the NAEP index (which was the sum of the 4th grade and 8th grade 2010 Math and Reading scores for each state). Notice that the relationship between the state-level SAT scores and the NAEP index jumps all over the place. In fact the formal linear correlation between the SAT and the NAEP index is 0.03 (R-squared = 0.00). Knowing the SAT score tells you essentially nothing about the likely NAEP index score.
Full 50-state data
Key Data: 50 States with % Sr taking SAT, combined SAT 2011, NAEP 2010 index, % 18 and under classified as in poverty. Sorted by NAEP Index high to low
Several southern states shown in bold
State |
% Sr. taking SAT |
|
|
Children in Poverty |
| Massachusetts |
97% |
1,549 |
1,065 |
14% |
| New Jersey |
90% |
1,508 |
1,048 |
14% |
| New Hampshire |
85% |
1,559 |
1,046 |
10% |
| Vermont |
72% |
1,538 |
1,041 |
17% |
| Maryland |
83% |
1,492 |
1,037 |
13% |
| Minnesota |
8% |
1,778 |
1,037 |
15% |
| Montana |
27% |
1,592 |
1,035 |
20% |
| Connecticut |
97% |
1,535 |
1,032 |
13% |
| North Dakota |
3% |
1,759 |
1,031 |
16% |
| Colorado |
20% |
1,699 |
1,031 |
17% |
| Virginia |
78% |
1,516 |
1,028 |
14% |
| Kansas |
7% |
1,734 |
1,027 |
18% |
| Pennsylvania |
86% |
1,473 |
1,027 |
19% |
| Wyoming |
5% |
1,692 |
1,026 |
14% |
| Maine |
98% |
1,391 |
1,025 |
18% |
| Ohio |
23% |
1,606 |
1,025 |
23% |
| Wisconsin |
5% |
1,767 |
1,022 |
19% |
| South Dakota |
4% |
1,737 |
1,021 |
18% |
| Washington |
61% |
1,560 |
1,019 |
18% |
| Kentucky |
6% |
1,711 |
1,017 |
26% |
| Idaho |
21% |
1,598 |
1,016 |
19% |
| North Carolina |
77% |
1,475 |
1,015 |
25% |
| Delaware |
93% |
1,455 |
1,014 |
18% |
| Indiana |
75% |
1,469 |
1,014 |
22% |
| Nebraska |
5% |
1,745 |
1,014 |
18% |
| Utah |
6% |
1,667 |
1,013 |
16% |
| Iowa |
4% |
1,777 |
1,013 |
16% |
| Rhode Island |
82% |
1,477 |
1,012 |
19% |
| Texas |
64% |
1,446 |
1,011 |
26% |
| Missouri |
5% |
1,764 |
1,010 |
21% |
| Illinois |
6% |
1,807 |
1,007 |
19% |
| New York |
95% |
1,460 |
1,006 |
21% |
| Florida |
75% |
1,447 |
1,004 |
23% |
| Oregon |
52% |
1,540 |
1,001 |
22% |
| Michigan |
5% |
1,760 |
1,000 |
23% |
| Georgia |
81% |
1,445 |
1,000 |
25% |
| South Carolina |
71% |
1,436 |
994 |
26% |
| Arkansas |
5% |
1,692 |
992 |
28% |
| Oklahoma |
6% |
1,683 |
992 |
25% |
| Alaska |
47% |
1,513 |
989 |
13% |
| Hawaii |
78% |
1,448 |
988 |
14% |
| Arizona |
40% |
1,539 |
987 |
24% |
| Nevada |
57% |
1,460 |
986 |
22% |
| Tennessee |
11% |
1,710 |
981 |
26% |
| Alabama |
9% |
1,622 |
979 |
28% |
| West Virginia |
17% |
1,512 |
978 |
25% |
| California |
58% |
1,513 |
973 |
22% |
| New Mexico |
13% |
1,618 |
971 |
30% |
| Louisiana |
8% |
1,651 |
969 |
27% |
| Mississippi |
4% |
1,660 |
962 |
33% |