Promise Academy Charter Middle School in Harlem - Social Programs Social Programs That Work

Evidence Rating:

Near Top Tier

Highlights

Program:

A charter middle school in New York City, serving mainly low-income, minority students.
Evaluation Methods:

A well-conducted randomized controlled trial (RCT), based on the lottery used to determine which students were offered admission, with a sample of 599 rising 6th graders.
Key Findings:

In long-term follow-up, students offered admission to Promise Academy (i) were significantly more likely to graduate high school on time than the control group (71% versus 58%), and (ii) scored significantly higher than the control group on New York Regents exams in math, English, and social studies (the effects equate to about 1-2 additional years of learning between 6th and 12th grade).
Other:

A study limitation is that it was conducted in a single site – one school in New York City. Replication of these findings in a second trial, in another setting, would be desirable to confirm the initial results and establish that they generalize to other settings where the program might be implemented.

The Promise Academy is a charter middle school, serving predominantly low-income, minority students from grades six through eight. The school opened in 2004 as one of the Harlem Children’s Zone programs to improve communities and schools in a 97-block area of Harlem, in New York City. The school provides an extended school day and year, with coordinated after-school tutoring and additional Saturday classes for children struggling in math or English language arts. As a result, students spend 50-100% more time in school per year than students in traditional public schools in New York City, depending on how far behind they are academically. The school emphasizes recruiting and retaining high quality teachers, who are incentivized and evaluated based on their success in raising students’ test scores. Students are consistently reminded of the importance of hard work in achieving success, and are given rewards for achievement, such as money or trips. The school also provides them with free medical, dental, and mental health services; and provides their parents with meals, bus fare, and other benefits. Most students who attend Promise Academy middle school subsequently enroll in a Promise Academy high school. The school spent approximately $22,300 per pupil per school year, compared to an average of approximately $18,700 per pupil in New York City middle schools (in 2017 dollars). [1]

Promise Academy’s website is linked here.

To see our full evidence summary:

Download PDF

Top Tier

Programs shown in well-conducted RCTs, carried out in typical community settings, to produce sizable, sustained effects on important outcomes. Top Tier evidence includes a requirement for replication – specifically, the demonstration of such effects in two or more RCTs conducted in different implementation sites, or, alternatively, in one large multi-site RCT. Such evidence provides confidence that the program would produce important effects if implemented faithfully in settings and populations similar to those in the original studies.

Near Top Tier

Programs shown to meet almost all elements of the Top Tier standard, and which only need one additional step to qualify. This category primarily includes programs that meet all elements of the Top Tier standard in a single study site, but need a replication RCT to confirm the initial findings and establish that they generalize to other sites. This is best viewed as tentative evidence that the program would produce important effects if implemented faithfully in settings and populations similar to those in the original study.

Suggestive Tier

Programs that have been evaluated in one or more well-conducted RCTs (or studies that closely approximate random assignment) and found to produce sizable positive effects, but whose evidence is limited by only short-term follow-up, effects that fall short of statistical significance, or other factors. Such evidence suggests the program may be an especially strong candidate for further research, but does not yet provide confidence that the program would produce important effects if implemented in new settings.

References

[1] Both figures are direct service expenditures per pupil, which is a comprehensive measure of expenditures for services provided directly to students during the school year. The source for the Promise Academy estimate is Dobbie et al. 2010. The source for the city average is the New York City Department of Education’s School Based Expenditure Reports.

Dobbie, Will and Roland G. Fryer, Jr., “The Medium-Term Impacts of High Achieving Charter Schools,” Journal of Political Economy, vol. 123, no. 5, 2015, pp. 985-1037.

Dobbie, Will and Roland G. Fryer, Jr., “Are High Quality Schools Enough to Increase Achievement Among the Poor? Evidence from the Harlem Children’s Zone,” American Economic Journal: Applied Economics, vol. 3, no. 3, July 2011, pp. 158-187.

Bloom, Howard S., Carolyn Hill, Alison Rebeck Black, and Mark Lipsey, “Performance Trajectories and Performance Gaps as Achievement Effect-Size Benchmarks for Educational Interventions,” MDRC Working Paper on Research Methodology, October 2008.

Orr, Larry L., Social Experimentation: Evaluating Public Programs with Experimental Methods, Sage Publications, Inc., 1999, pp. 62-64.

Bloom, Howard S., “Accounting for No-Shows in Experimental Evaluation Designs,” Evaluation Review, vol. 8, April 1984, pp. 225-246.

Social Programs That Work

Promise Academy Charter Middle School in the Harlem Children’s Zone

Highlights

Program:

Evaluation Methods:

Key Findings:

Other:

References

SUBSCRIBE TO OUR NEWSLETTER

Promise Academy Charter Middle School in the Harlem Children’s Zone

Highlights

Program:

Evaluation Methods:

Key Findings:

Other:

References

Related Reviews

KIPP Charter Schools

Career Academies

New York City’s Small Schools of Choice

SUBSCRIBE TO OUR NEWSLETTER