- Intervention: A charter middle school in New York City, serving mainly low-income, minority students.
- Evaluation Methods: A well-conducted randomized controlled trial, based on the lottery used to determine which students were offered admission, with a sample of 599 rising 6th graders.
- Key Findings: In long-term follow-up, students offered admission to Promise Academy (i) were significantly more likely to graduate high school on time than the control group (71% versus 58%), and (ii) scored significantly higher than the control group on New York Regents exams in math, English, and social studies (the effects equate to about 1-2 additional years of learning between 6th and 12th grade).
- Other: A study limitation is that it was conducted in a single site – one school in New York City. Replication of these findings in a second trial, in another setting, would be desirable to confirm the initial results and establish that they generalize to other settings where the program might be implemented.
EVIDENCE RATING: NEAR TOP TIER
The standard for Near Top Tier is:
Programs shown to meet almost all elements of the Top Tier standard, and which only need one additional step to qualify. This category primarily includes programs that meet all elements of the Top Tier standard in a single study site, but need a replication RCT to confirm the initial findings and establish that they generalize to other sites. This is best viewed as tentative evidence that the program would produce important effects if implemented faithfully in settings and populations similar to those in the original study.
DESCRIPTION OF THE PROGRAM
The Promise Academy is a charter middle school, serving predominantly low-income, minority students from grades six through eight. The school opened in 2004 as one of the Harlem Children’s Zone programs to improve communities and schools in a 97-block area of Harlem, in New York City. The school provides an extended school day and year, with coordinated after-school tutoring and additional Saturday classes for children struggling in math or English language arts. As a result, students spend 50-100% more time in school per year than students in traditional public schools in New York City, depending on how far behind they are academically. The school emphasizes recruiting and retaining high quality teachers, who are incentivized and evaluated based on their success in raising students’ test scores. Students are consistently reminded of the importance of hard work in achieving success, and are given rewards for achievement, such as money or trips. The school also provides them with free medical, dental, and mental health services; and provides their parents with meals, bus fare, and other benefits. Most students who attend Promise Academy middle school subsequently enroll in a Promise Academy high school. The school spent approximately $22,300 per pupil per school year, compared to an average of approximately $18,700 per pupil in New York City middle schools (in 2017 dollars).1
Promise Academy’s website is linked here.
EVIDENCE OF EFFECTIVENESS
Our summary for the evidence of this program is linked here.
- Dobbie, Will and Roland G. Fryer, Jr., “The Medium-Term Impacts of High Achieving Charter Schools,” Journal of Political Economy, vol. 123, no. 5, 2015, pp. 985-1037.
- Dobbie, Will and Roland G. Fryer, Jr., “Are High Quality Schools Enough to Increase Achievement Among the Poor? Evidence from the Harlem Children’s Zone,” American Economic Journal: Applied Economics, vol. 3, no. 3, July 2011, pp. 158-187.
- Bloom, Howard S., Carolyn Hill, Alison Rebeck Black, and Mark Lipsey, “Performance Trajectories and Performance Gaps as Achievement Effect-Size Benchmarks for Educational Interventions,” MDRC Working Paper on Research Methodology, October 2008.
- Orr, Larry L., Social Experimentation: Evaluating Public Programs With Experimental Methods, Sage Publications, Inc., 1999, pp. 62-64.
- Bloom, Howard S., “Accounting for No-Shows in Experimental Evaluation Designs,” Evaluation Review, vol. 8, April 1984, pp. 225-246.
1 Both figures are direct service expenditures per pupil, which is a comprehensive measure of expenditures for services provided directly to students during the school year. The source for the Promise Academy estimate is Dobbie et al. 2010. The source for the city average is the New York City Department of Education’s School Based Expenditure Reports.