← Data Science for Educators

Charter School Penetration and NAEP Scores

Do states with more charter schools have higher or lower NAEP scores, after controlling for demographic composition?

Comparative → Predictive

What is this question really asking?

Charter school enrollment varies enormously by state — from near zero in West Virginia to over 15% in Arizona. The question is whether higher charter penetration is associated with better state-level NAEP scores after accounting for differences in poverty, race, and special education rates. Without those adjustments, you might credit (or blame) charter schools for differences that are really about which students live in which states.

Why classification matters. This is comparative across states and predictive because charter share is used to model expected NAEP scores. It is not automatically causal — states with lots of charters might differ in other ways that drive scores. Causal claims require additional design, like an instrumental variable.

How you would investigate it

  1. Make a scatterplot. Put charter enrollment percentage on the x-axis and NAEP math scores on the y-axis — each state is one dot. A visible upward or downward slope tells you there is a raw relationship worth investigating.

  2. Add fairness controls. Run an OLS regression: NAEP score = charter share + % Black + % Hispanic + % free lunch + % IEP + per-pupil spending + region. The regression answers: if two states had the same demographics and funding levels, would the one with more charter schools score higher?

  3. Address the chicken-and-egg problem. Did charters cause higher scores, or did low-scoring states adopt charters as a reform? Use an instrumental variable: the year a state passed its charter law affects charter penetration today but is not caused by current demographics. If charter penetration predicted by law passage year is associated with higher scores, the case for causation strengthens.

  4. Look for natural experiments. When a state lifts its charter school cap, compare before-and-after score changes to states that did not lift their caps. This difference-in-differences design isolates the policy effect.

Data you would need

What Source Example
State NAEP scores, 2019/2022/2024 NAEP Data API MA: 283, AZ: 270
State demographics NAEP Data API % Black, % Hispanic, % free lunch, % IEP
Charter enrollment % by state NCES Common Core of Data AZ: 18%, WV: 0%
Charter quality ratings CREDO (Stanford) Ohio charters: above avg. reading
State charter law history Education Commission of the States MN: first law, 1991
Per-pupil expenditure U.S. Census Bureau NY: $25K/student; UT: $9K/student

NAEP API variables: TOTAL, MN:MN, SDRACE, SLUNCH3, IEP, CHRTRPT, C0863A1–C0863I1

Analytic method: State-level OLS regression with region fixed effects. Multiple years pooled. Robustness: instrumental variable using charter law passage year; difference-in-differences exploiting state-level charter cap changes.