The Promise and Reality of Social Impact Bonds
In March of 2015, The Rockefeller Foundation announced “tangible success” for a social impact bond supporting a program designed to reduce recidivism in Peterborough in the United Kingdom. The program resulted in a four percent reconviction differential between experimental and control groups of released prisoners. The foundation said the Peterborough experiment “launched global excitement about the social impact bond model.”1 But since the initial rush of excitement, appropriate caution and skepticism has grown about social impact bonds as a tool to produce innovation and to get significant amounts of private capital dedicated to solving difficult social problems.
Social impact bonds, or “Pay for Success” investments, continue to draw much attention and hope in the philanthropic, social services, and public sectors. Social impact bonds are designed to link private, for-profit capital with social- and public-sector innovation, stimulating program innovation and providing a new source of risk-taking capital for the resource-starved public sector. Muhammad Yunus, founder of the Grameen Bank, and Judith Rodin, former president of The Rockefeller Foundation, summarized this dual attraction succinctly in an article entitled “Save the World, Turn a Profit.”2
But in practice, social impact bonds are showing limited results. Although some advocates argue that early stage social impact bonds will work through their initial challenges,3 we think the limited results to date suggest more basic flaws. We are concerned that many advocates for social impact bonds misdiagnose why the public sector is slow to support innovation, and they also misunderstand how for-profit capital seeks a balance of risk and return. We worry, too, that sincere social impact bond advocates are diverting attention, resources, and creative talent that could better address the pressing social problems they care about.
THE GROWTH OF SOCIAL IMPACT BONDS: RHETORIC AND REALITY
The attention paid to social impact bonds has grown dramatically since their first successful deployment in the Peterborough anti-recidivism program. In the past few years, McKinsey, Accenture, Deloitte, the Brookings Institution, Goldman Sachs, the Wharton School, the Conference Board, the Stanford Business School, New York University, and many others have issued favorable reports on social impact bonds. The United Kingdom established a Cabinet Office of Social Impact Bonds, the Obama administration endorsed what it called Pay for Success, and there have been congressional hearings that advocate for legislation. There are annual conferences and panels that focus on social impact bonds, and scholars, advocates, foundation staff, and private investors have written papers on the subject.
But although there are many advocates, there seem to be relatively few deals, with existing ones highly concentrated in the United Kingdom and to a lesser extent in the United States.4 And it is not clear how many of these deals really are just standard pay-for-performance contracts with nonprofits or government agencies.5
Given the excitement around social impact bonds, observers might not realize that they rest on a very small evidence base of programmatic success. The iconic social impact bond case is the Peterborough, England, recidivism reduction program, which initially reduced recidivism by 8.4 percent relative to a control group. However, the program had targeted a ten percent reduction in order to pay investors immediately, although the program was on track to reach its average 7.5 percent reduction over time. But the United Kingdom phased out the program in 2015 due to larger changes in rehabilitation policy.
In the United States, Goldman Sachs invested in a program at New York City’s Rikers Island jail, making a $9.6 million loan to fund targeted therapy for 16- to 18-year-olds, with Bloomberg Philanthropies guaranteeing 83 percent of Goldman Sachs’ investment against losses. The program was rigorously evaluated against a goal of reducing recidivism by at least ten percent. When the young people going through the program showed no significant reductions in recidivism it was canceled. Although there was no direct budgetary cost to New York City, there were operating disruptions from introducing the program, and costs born by the Bloomberg Foundation’s $6 million guarantee of over 80 percent of Goldman Sach’s investment.
Goldman Sachs, along with the Pritzker Family Foundation, also invested in a Utah early childhood education program often cited as a success based on an unusually high finding that 109 of 110 participants were successfully diverted from future expensive special education. This finding is far out of line with other similar programs for poor children.6 Yet this triggered an initial payment to Goldman Sachs, with a rate of return estimated between five and seven percent.7
WHAT PROBLEMS ARE SOCIAL IMPACT BONDS TRYING TO SOLVE?
Social impact bond advocates seek progress on four major issues. They believe that social impact bonds can:
- Spur program innovation. Advocates argue that social impact bonds will encourage greater exploration of existing or potential social innovations. Using private investment to incentivize nonprofits and governments will allow new innovations to be tested and then scaled up, as they are in the private market.
But private-sector innovations funded by venture capital mostly fail, at rates ranging from 65 to 90 percent.8 Faced with that reality, some social impact bond advocates now argue that the investment risk should be reduced by relying on some earlier “proof of concept” by a nonprofit, often funded by philanthropy. That is an appropriate role for foundations, but it isn’t clear why for-profit social impact bonds then are necessary for governments to scale up early program success.
Some advocates argue that foundations are taking on the risky venture capital role by funding this first stage, but that undercuts the claim that social impact bonds can bring significant risk-tolerating private capital into the social innovation space. If deals cannot be structured to attract private capital to take high risk for high returns, the social impact bond model is not scalable. - Improve nonprofit and government operating performance. By bringing in private capital, advocates argue that nonprofits and government agencies will perform better because of market discipline and oversight.
Complaints about governmental lack of innovation are longstanding and, in our generation, go back at least to the “Reinventing Government” movement driven by then-Vice President Al Gore, which sought to adapt “best practices” from the private sector. But government is not a private business, and there are questions as to what extent government can or should operate like a private firm. Government can learn from private firms (and vice versa), but we should be well past the day when we invoke private-market efficiency as a simple, broad solution to public management problems. - Attract private capital. Advocates see social impact bonds as a way to get profit-seeking private capital into public and nonprofit work, arguing that this will increase resources in a time of public austerity. But critics argue that social impact bonds work at a very small financial scale, so there is no significant overall budget relief. And private-sector investors seek to minimize risk and uncertainty, not create or embrace it. As in the Rikers Island case, investors will seek guarantees or other risk-minimizing practices such as introducing more complex and differentiated payment tranches to protect their investment, cutting against the social impact bond argument that private capital will increase government’s risk tolerance.9
Operationally, social impact bonds demand a lot of administrative attention, added expenses for monitoring and evaluation, and budgeting for contingent liabilities created by promises of future repayments to investors. Adding them into a regular government program and budgeting structure can create real disruptions while getting the benefits of only a very small additional investment. - Use rigorous evaluation methods. Social impact bond advocates often promise a strong evidence base in the form of rigorous random assignment evaluations, where demonstrated success is required before investors are paid. But high-quality rigorous evaluations are expensive and take a long time to yield results. And the methodology militates against changing program models during the evaluation, which paradoxically means that social-impact-bond-funded programs threaten the rigor of their evidence if they try to learn and improve their operations.
Government programs, especially new innovations, need flexibility in the early stages. The Rikers Island program was based on scaling up a program for 16- to 18-year-olds. However, not enough participants could be found in those age groups, so older youth were added, and the program that was delivered differed significantly from the original “proof of concept” version. While these changes may have been sensible adaptations to the operational reality at Rikers, they challenged the evaluation design and success benchmarks. Some advocates claim that the Rikers program was unlike other programs so program problems and evaluation difficulties stemmed from its novelty, but there is a body of evaluation literature on such programs in a variety of contexts that undercuts that claim.10
And faced with the pressure to produce findings, evaluation rigor can easily slip as investors seek repayment and program advocates want to claim success. The Utah early childhood program payouts to Goldman Sachs were based on problematic assessments claiming success far beyond any impacts ever found for similar programs, and also were not based on the type of rigorous evaluation that social impact bond advocates often endorse.
While we would not insist on RCT evaluations (because of their cost and how they impose a lack of flexibility on programs), the claim of 99 percent success should have raised a caution flag and been examined more critically. Advocates for the Utah program generally do not discuss how such an extraordinary success level (99 percent) was achieved. Nor do they compare their very high level of reported success to other rigorous evaluations of how early childhood programs for low-income children reduce later referrals to special education. There are a number of rigorous studies that find a positive impact, but at much lower magnitudes than the Utah program reports.
SOCIAL IMPACT BONDS: SOLUTION OR DISTRACTION?
We strongly agree with social impact bond advocates that more program innovation, improved government and nonprofit operational performance, increased funding, and better assessment are necessary to address major social problems. But we believe each of those goals can be addressed through other means, and that trying to find one “magic bullet” solution through social impact bonds may actually weaken results for all four goals.
First, philanthropy can and should continue to encourage and test innovative program ideas. But these can be financed through grants, or in the case of programs with a revenue stream, recoverable grants or program-related investments (PRIs). Putting private-equity investors or banks into this space may well weaken innovation instead of encourage it, as for-profit investors generally seek some assurance of positive financial returns and discourage risk in the absence of high expected returns.
Second, government and nonprofit agencies do need improved performance. But that may be better done through encouraging “better practices” in public management, or through such models as The Robin Hood Foundation, where grantees are provided with a high level of technical assistance and deep managerial engagement to improve operations. Technical assistance to grantees, or changed public management practices, will not suddenly appear or be implemented because of social impact bonds. And any additional costs for technical assistance will increase total program expenditures, raising the per capita costs and lowering the return on investment.
Finally, we strongly agree that increased funding is needed for testing innovations and bringing them to scale. But the testing can be done by philanthropy, and scaling should come through government’s existing ability to tax, spend, and evaluate. Money being spent on the mechanics of social impact bonds might better be spent on advocacy for appropriate levels of taxation to meet our social needs rather than trying to leverage for-profit capital into difficult non-market problems.
And we see some worrisome signs about social impact bonds. For example, there is no justification for exploiting philanthropy’s tax-advantaged status to guarantee private profits, as in the Rikers Island case, where Bloomberg Philanthropies (itself already benefiting from its tax-free status) guaranteed around 75 percent of Goldman Sachs’ investment returns.11 Using public tax expenditures in that way wastes scarce resources and reduces accountability.
Perhaps our biggest concern is that social impact bonds and other Pay for Success approaches are drawing too much philanthropic, governmental, and nonprofit talent and creativity at the cost of other support for social innovation. Philanthropy, government, and nonprofits need to keep at the difficult but necessary work of improving program outcomes in fields like criminal justice and education, and support the advocacy and coalition work needed to increase funding for public programs. There is no private-sector-based magic that will solve these critical needs, and the money, talent, and energy devoted to social impact bonds might better be directed elsewhere.
1 The Rockefeller Foundation, “Peterborough Social Impact Bond Reduces Reoffending by 8.4%: Investors on Course for Payment in 2016” (March 16, 2015), available at https://www.rockefellerfoundation.org/report/peterborough-social-impact-bond-reduces-reoffending-by-8-4-investors-on-course-for-payment-in-2016/.
2Muhammad Yunus and Judith Rodin, “Save the World, Turn a Profit,” Bloomberg View (September 25, 2015), available at https://www.bloomberg.com/view/articles/2015-09-25/save-the-world-turn-a-profit.
3 Kenneth Dodge, “Why Social Impact Bonds Still Have Promise,” The New York Times (November 13, 2015), available at http://www.nytimes.com/2015/11/14/business/dealbook/why-social-impact-bondsstill-have-promise.html.
4It is difficult to get a consistent count of how many social impact bonds are in existence, due to a combination of varying definitions of what qualifies, whether to count a large fund as one social impact bond or count each locality using that fund as a separate social impact bond, whether the count contains only active projects or also those in development, etc.
5Some social impact bond advocates now argue that calling them “bonds” is misleading, because of variable and risky returns that make the investments more like equity: Lindsay Beck et al., “Social Impact Bonds: What’s in a Name?” Stanford Social Innovation Review (October 12, 2016), available at https://ssir.org/articles/entry/social_impact_bonds_whats_in_a_name. But the vehicle — whatever it is called — still is intended to get significant amounts of profit-seeking capital into social innovation.
6The best known programs for low-income children that have been rigorously studied are the High/Scope Perry Preschool, Abecedarian, and Child Parent Center (CPC) programs. None of them, or others, found effects anywhere close to those reported for the Utah program. See the review in W. Steven Barnett, “Preschool Education and Its Lasting Effects: Research and Policy Implications,” National Institute for Early Education Research at Rutgers University (September 2008), available http://nepc.colorado.edu/files/PB-Barnett-EARLY-ED_FINAL.pdf.
7The evaluation also was not done with a random assignment methodology, and used diagnostic instruments that some researchers claim could have resulted in very large misclassifications of children’s potential entry into special education: Nathaniel Popper, “Success Metrics Questioned in School Program Funded by Goldman,” The New York Times (November 3, 2015), available at http://www.nytimes.com/2015/11/04/business/dealbook/did-goldman-make-the-grade.html?_r=0.
8See among others: CB Insights, “The Venture Capital Funnel: Your Chances of Raising Follow-Ons, Existing, and Becoming a Unicorn” (December 9, 2015); Deborah Gage, “The Venture Capital Secret: 3 Out of 4 Start-Ups Fail,” The Wall Street Journal (September 20, 2012), available at https://www.wsj.com/articles/SB10000872396390443720204578004980476429190.
9McKinsey & Company, “Taking Off: A Hybrid Investment Fund to Unlock the Growth Potential of Social Enterprises in Germany” (December 2016), available at https://www.mckinsey.de/files/170109_report_impact_investment_hybrid_fund.pdf; Liz Farmer, “New ‘Pay for Success’ Model Offers Money-Back Guarantee,” Government Technology (March 25, 2016), available at http://www.govtech.com/budget-finance/Pay-for-Success-Programs-Success-or-Your-Money-Back.html.
10Nana A. Landenberger and Mark W. Lipsey, “The Positive Effects Of Cognitive-Behavioral Programs For Offenders: A Meta-Analysis Of Factors Associated With Effective Treatment,” Journal of Experimental Criminology 1(4) December 2005: 451–476; On moral reconation therapy in particular, see L.M. Ferguson and J.S. Wormith, “A Meta-Analysis of Moral Reconation Therapy,” International Journal of Offender Therapy and Comparative Criminology 57(9) 2013: 1076–1106.
11Goldman Sachs then wrote off the nonguaranteed part of the investment, presumably also deducting those losses from its tax bill. So there are significant tax subsidies undergirding all of the Goldman Sachs investment, which seems to be another hidden cost not accounted for when calculating the total cost of social impact bonds.