No. 24 December 2002
2001 Education Freedom Index
Jay P. Greene, Senior Fellow, The Manhattan Institute for Policy Research
The Education Freedom Index measures the extent of government-subsidized or -regulated educational choices offered to families in each state. The author also analyzes the relationship between the amount of education freedom in a state and the level of academic achievement demonstrated by its students.
Specifically, this report finds that:
- Arizona offers parents the most educational freedom, followed by New Jersey, Delaware and Florida. Hawaii offers the least educational freedom, followed by Utah, Rhode Island and West Virginia;
- Academic achievement is positively correlated with educational freedom. Indeed, if a state could improve its Education Freedom Index Score by one point, we would expect that an additional 4.1% of its students would perform proficiently on the NAEP math test.
ABOUT THE AUTHOR
Jay P. Greene is a senior fellow at the Manhattan Institute for Policy Research where he conducts research and writes about education policy. He has conducted evaluations of school choice and accountability programs in Florida, Charlotte, Milwaukee, Cleveland, and San Antonio. He has also investigated the effects of school choice on civic values and integration.
His articles have appeared in policy journals, such as The Public Interest, City Journal, and Education Next, as well as in academic journals, such as The Georgetown Public Policy Review, Education and Urban Society, and The British Journal of Political Science. Most recently he published a critique of the GED diploma in the Winter issue of City Journal and an evaluation of the Florida A-Plus choice and accountability system in the Winter issue of Education Next. He also recently authored two chapters in Charters, Vouchers, and Public Education published by the Brookings Institution.
Dr. Greene has been a professor of government at the University of Texas at Austin and the University of Houston. He received his Ph.D. from the Government Department at Harvard University in 1995. He lives with his wife and three children in Weston, Florida.
THE EDUCATION FREEDOM INDEX
State governments do not, in general, restrict the liberties of families to raise their children—do not prescribe what clothes they should wear, what food they should eat, what books they should read, what ideas they should discuss over dinner. Our states do vary widely, however, in the degree of freedom they give parents to decide how their children learn math, literature, science, or history. The traditional justification for restricting choice in education has been the assertion of a compelling public interest. The evidence presented here, however, suggests a compelling public interest not in limiting education freedom, but in increasing it. Parental choice leads to more efficient educational outcomes.
In September 2000 the Manhattan Institute released the Education Freedom Index, which measured the extent of government-subsidized or -regulated educational choices offered to families in each state. That report also contained statistical analyses that suggested a relationship between the amount of education freedom in a state, and the level of academic achievement demonstrated by its students. This second edition of the Index documents significant changes regarding the choices offered to families. This year’s report also reaffirms the relationship between education freedom and academic achievement, adding some improvements to the statistical model. The results are good news for those who believe that parental control over the education of their children is central to good education policy.
Changes in the Method
One important change has been made in the method used to calculate education freedom. Last year five components of the Education Freedom Index (EFI) captured the types of choices offered to families: charter schools, subsidized private schools, home-schooling, relocation (the ease of moving to another district), and inter-district transfer. This year, the later two categories have been combined to form a single public-school choice-component. The combination of these two components, motivated by conceptual and practical concerns,1 has a relatively small influence on the scores and rankings of the states, as can be seen in Appendix Table A.
The computation method otherwise remains as in last year’s report. The Education Freedom Index score, then, is the average of these four components:
- Charter-School Choice. This year, using more recent data, we measure the extent of charter-school choice by the percentage of all public schools that are charter schools in each state. We also measure the extent to which the state has a charter school law that facilitates charter school choice. Both of those measures are standardized and then averaged to compute the charter-school choice score.
- Subsidized Private-School Choice. The score for subsidized private school choice continues to consist of the average of three standardized measures: the percentage of students enrolled in voucher programs, the extent of a tax credit for private school expenses, and the availability of direct subsidies to private schools for certain expenses.
- Home-Schooling Choice. The home-schooling score consists of the percentage of students home-schooled in each state, which has not been updated since last year’s EFI, and an updated ranking of each state’s regulatory environment for those choosing home-schooling compiled by the Home School Legal Defense Association.
- Public-School Choice. The new component averages three updated, standardized measures: 1) the average square miles of districts in each state, 2) the average population in districts in each state (both of these measure how disruptive to a family it would be to move to a different school district, considering distance and commuting times to old jobs, family, and friends from another district), and 3) a three-point scale taken from Education Week’s Quality Counts report that measures the availability of inter-district transfer options. The scale indicates whether states permit inter-district transfers, limited transfers, or no tranfers.
Highlights and Lowlights
Arizona again reigns supreme. Its number one ranking is largely attributable to a large range of charter-school options and to a tax credit for private-school scholarships. (See Table 1).
New Jersey, which moved up slightly to 2nd place, has strengths across the board. It has many small districts, allowing families to move to desired school districts; it offers a wide selection of charter school options; and it offers direct subsidies to private schools for certain expenses. New Jersey is also relatively accommodating to home-schooling.
Delaware, moving up 4 notches to 3rd place, has a particularly large selection of charter schools relative to its total number of schools, and shows solid results in all other components.
For the second year in a row, Hawaii is at the very bottom. Because the state has only one school district, Hawaiian parents cannot exercise public school-choice by moving to other districts or by transferring their children across district lines. And while Hawaii has improved its charter-school options, it still regulates home-schooling fairly heavily, and offers no subsidies for private-school choice.
Utah—which offers no assistance for private school choice, closely regulates home-schooling, and has large school districts and weak charter-school offerings—slipped to second-to-last place. Rhode Island, because of similar restrictions, comes in 48th.
Breakdown by Components
Breaking the rankings down by each of the components reveals which states excel or lag in specific spheres.
- Charter-School Options. Arizona, Delaware, and Michigan earn the highest marks. (See Table 2) Each offers a large selection of charter schools relative to the total number of schools in the state, and each has a regulatory environment that is favorable to the creation and operation of charter schools. Thirteen states share the lowest rating in this category because they offer no charter schools.
- Subsidies for Private-School Choice. Overall, Florida, Maine, and Vermont earn the top three spots for subsidized private school choice. (See Table 3) The thirteen states that offer no assistance for private school choice share last place. In most of the 37 states which offer some assistance, the government provides direct subsidies to private schools for textbooks, transportation, or health and “auxiliary services.” Six states—Arizona, Florida, Illinois, Iowa, Minnesota, and Pennsylvania—offer tax credits for private-school expenses or scholarships. The Florida and Pennsylvania laws, adopted since the last EFI report was written, show the trend toward this type of education freedom. Five states, Florida, Maine, Ohio, Vermont, and Wisconsin, offer voucher programs that pay tuition at private schools. The Maine and Vermont programs have been in existence for over a century and are known as “tuitioning” programs. The programs in the other three states continue to grow and assist low-income or special needs children in choosing a private school.
- Regulatory Environment for Home-Schooling. Five states—Idaho, Missouri, New Jersey, Oklahoma, and Texas—share the highest ranking. (See Table 4) Massachusetts, Rhode Island, and Utah, due to their heavy regulation of home-schooling, share the lowest mark.
- Public-School Choice. Nebraska, Oklahoma, and Iowa, each of which offers inter-district transfer opportunities and has relatively small school districts, are the top three. (See Table 5) Hawaii, Alaska, and Nevada, with their relatively large school districts and their lack of inter-district transfer programs, are at the bottom.
Changes in Rankings
Because most states did not experience major policy changes over the past year, most states’ EFI rankings did not dramatically change (see Table 1). Those states that did enact major changes in policy, however, experienced significant changes in their standings.
Florida, which jumped from 35th to 4th place, has shown the greatest improvement. That improvement is largely related to Florida’s rapid expansion of government-subsidized private-school choices. Its McKay Scholarship Program, which provides especially needy families with private-school vouchers, is rapidly becoming one of the largest such programs in the country. Florida has also adopted an extensive tax-credit program for contributions to private-school scholarship funds that assist low-income families. In addition, chronically failing schools offer vouchers as part of Florida’s A-Plus accountability program. On top of all that, Florida has continued to widen charter-school options.
Other states showing significant improvement include Oklahoma and Indiana, which increased their rankings by 21 and 12 places, respectively. Both states recently introduced charter schools, and continued to offer accommodating regulatory environments for home-schooling. Missouri, North Carolina, and Pennsylvania each moved up 11 places; Missouri and North Carolina received better grades from the Home School Legal Defense Association, while Pennsylvania adopted a new tax credit for private school scholarship funds.
Just as some states moved ahead in the EFI rankings, others slipped back. Those states that declined did so, for the most part, not because they became more restrictive of educational options, but because they failed to expand those options as rapidly as other states.
Utah, for example, slipped 20 rankings to come in 49th place. That decline resulted not only from the lower grade Utah received from the Home School Legal Defense Association, but from the state’s failure to expand charter-school options while other states were rapidly doing so. For similar reasons, Iowa fell 19 places, to 33rd. South Dakota dropped 16 places, to 31st, largely because it continued to lack charter schools and subsidies for private-school choices, and thus fell behind states which initiated or expanded those options.
Education Freedom Improves Student Achievement
As in last year’s EFI report, we have examined the statistical relationship between (a) the Index score in each state, and (b) the average level of achievement of students in that state, after controlling for certain other factors that may influence student achievement.
This statistical relationship can only be suggestive of an actual causal relationship, because our model is necessarily limited. There are only 50 states to consider, and several states lack test scores from the National Assessment of Educational Progress (NAEP), the best test-instrument for comparing state results over time. With a relatively small number of observations to include in the model, the statistical leverage that the model provides is necessarily modest. In addition, with a low number of observations we cannot possibly control for all other factors that we might want to include in a model of student achievement.
Despite these limitations, the observable relationship between education freedom and student achievement remains strong. Where families have more options in the education of their children, the average student tends to demonstrate higher levels of academic achievement.
Improvements in the Model
The statistical model has been improved in two ways. First, we have added a score for accountability systems, derived from The State of Standards 2000, released by the Thomas B. Fordham Foundation.2 Appendix G of that report shows whether a state has report cards for each school, whether successful schools are rewarded, whether a state rates the academic performance of each school, whether sanctions are threatened for failing schools, and whether sanctions are actually used. From these five data we have computed a scale to measure the thoroughness of each state’s accountability system. By adding a measure for the state’s accountability system, it is possible to compare the extent to which education freedom and state accountability each contribute to student achievement.
Second, we have added a control measure for the state’s test-score performance several years ago. If we find a significant relationship between education freedom and current test scores even after controlling for earlier test scores, we can be more confident that education freedom helped improve student performance, and was not simply an approach adopted by states that were already educationally successful for other reasons.
The model also controls for median household income, per-pupil spending, and the percentage of ethnic minorities in each state. Controlling for these other factors that may influence student achievement allows a more accurate picture of the relationship between education freedom and student achievement.
Improvements in Test Scores
The dependent variable in the model is the NAEP 8th grade math results from 2000, the most recent state test results available. The results, presented in Table 6, show that even after controlling for a number of other factors, there is a statistically significant relationship between each state’s EFI score, and the percentage of its students performing proficiently on the 8th grade math NAEP test. If a state could improve its EFI score by 1 point, we would expect that an additional 4.1% of students in the state would perform proficiently on the NAEP math test. But because the scores for the EFI are computed in standardized units (necessary to average across unlike units, such as averaging weight with height), the magnitude of a one-point change in the EFI score is not easy to grasp. To put a one-point improvement in the EFI in perspective, the difference between Arizona and Hawaii, the highest and lowest scoring states, is about 2 points. If we were to change nothing about Hawaii except to give it the same educational policies as existed in Arizona, we would expect about an additional 8% of students to perform proficiently on the NAEP math test.
Another way to put a one-point improvement in EFI score in perspective is to consider Florida’ progress. By adopting a tax credit for private school scholarships, expanding a voucher program for special needs students, and continuing to open new charter schools, Florida was able to increase its EFI score by about one-half point in one year’s time. According to the model in Table 6 we would expect Florida to enjoy an additional 2% of its students to perform proficiently on the NAEP math test as a result of its new policies.
Education Freedom is a Cost-Effective Means of Reform
In the regression model presented in Table 6 we also see that states with wealthier families, a smaller minority population, and high per pupil spending tend to experience higher NAEP math results. To compare the magnitude of these different influences, a one point increase in the EFI score would produce the same expected improvement in NAEP math results as increasing per pupil spending by $2,490, about a 36% increase in spending over the current national average. It is certainly cheaper to adopt policies that permit greater education freedom, as Florida did, than to try to realize similar test score gains simply by increasing per pupil spending. To produce the same expected gain in test scores as a one-point increase in education freedom we would need to increase the median household income in a state by $6,405. It would take quite a lot of wealth redistribution to achieve what could be accomplished by expanding education freedom.
At face value, the strength of a state’s accountability system does not seem to be significantly related to the math NAEP scores according to the model presented in Table 6. States with higher accountability scores, controlling for these other factors, do not have higher test scores. If we control for each state’s earlier test scores, however, we see that the strength of a state’s accountability system is significantly related to current test scores. (See Table 7)
States with initially low test scores but strong accountability systems can produce significant improvements in test scores, even if the states with strong accountability systems are not among the highest scoring states on the most recent test. According to the regression model shown in Table 7, a one-point increase in a state’s accountability score would lead to 1.1% more students scoring proficiently on the 2000 NAEP math test. A state can produce a one-point increase in the accountability score simply by threatening to sanction failing schools. A state can increase its accountability score by 2 points if it follows through on the threat. Producing a report card of school performance, rating that performance, and rewarding success can yield three more points in accountability score improvement.
Controlling for Earlier Test Scores
Controlling for early 1990s test scores in the model in Table 7 does not alter the relationship between education freedom and current test scores. States with the greatest education freedom are those that have experienced the greatest improvement in test scores and those that enjoy the highest current test scores, controlling for a number of other factors. The concern that states that already had the highest test scores were those that gravitated to education freedom does not seem to be supported by this analysis. Education freedom helped states improve their test scores.
For many years, education reformers have advocated strengthening accountability systems and expanding educational freedom. Our statistical models suggest that such reforms, where implemented, have yielded the academic improvements that reformers predicted.
Our models also demonstrate the cost-efficiency of improving education via expanded choice and strengthened accountability. Implementing tax credits for private school scholarships, adding new charter schools, adopting school report cards, sanctioning failure, and deregulating home-schooling can produce test score gains, in an entire state, that would otherwise require thousands of additional dollars in per-pupil spending.
- Conceptually, the ability to choose a different public school by moving to another school district or choosing a different school district by transferring are both types of traditional public school choice. Choosing a different public school district by relocating is by far the most common type of school choice and has been around as long as there have been school districts. Inter-district transferring by comparison is relatively new and few families participate in such programs, which leads to the practical reasons for combining these two types of public school choice into one category: there is very little information available on the extent of inter-district transferring. The only countrywide measure available is from the Quality Counts report issued by Education Week. It simply indicates whether states have inter-district transfer programs, have restricted programs, or have no programs at all. This three-point scale is not precise enough to deserve to be one-fifth of the entire EFI. By folding it into the public school choice category, with two other measures of relocational choice, the influence of this imprecise measure of inter-district transferring in the whole EFI is greatly reduced.
- I focus only on inter-district choice rather than intra-district choice because intra-district choice often has little effect on the incentives of schools to attend to the needs of students. That is, when students move from one district to another the financial support attached to that student also moves, providing schools with incentives to attract or retain the students in order to secure their funding. Moving from school to school within a district often does not have the same financial consequences for schools because their funding formulas are less reliant on per pupil formulas. Intra-district school choice therefore may help families find schools that satisfy their needs more effectively but such programs are not likely to alter the incentives of schools to attend to those needs. Similarly, moving from classroom to classroom within a school may help families find teachers they like better, but it does not affect the financial incentives of the educators. See Appendix G in the report at http://www.edexcellence.net/library/soss2000/2000soss.html