Scaling Programs That Work by Paying for Success
If parenthood is the toughest job you will ever love, then Pay for Success may be the most grueling growth strategy we will someday celebrate. We often get asked about Pay for Success and how to put together “a deal.” And we find that the best guidance we can provide is under the category of “lessons learned” or “things I wished I’d known.” As two experienced professionals with expertise in the nonprofit and governmental sectors, we often remind each other that these deals are among the most complicated and interesting (on good days) work that we have done.
Our goal with this essay is to honestly reflect on some of what we have learned with the hope that the paths become easier. And we start with a caveat: Because Pay for Success is labor-, and resource-intensive, we recommend that it be the “last and also the latest tool in the toolbox.” At this early stage in the development of this field, we urge policymakers not to use Pay for Success to replace existing funding streams but instead to supplement them. Otherwise, while deal construction is underway and when some deals fall apart, families will be left with fewer services and agencies will struggle with unpredictable futures.
At the same time, we are excited about ways to simplify Pay for Success and also use new innovations in outcomes-oriented funding to expand services and reach more people. We see in Pay for Success the potential to scale programs in a way that aligns funding with the outcomes orientation we have built over decades at Nurse-Family Partnership (NFP), a national program serving first-time, low-income mothers with home visits by certified nurses. The promise of unlocking new sources of aligned capital to expand services for the families and communities that need them is worth fighting for.
SHIFTING TO AN OUTCOMES MINDSET
Due to decades of ongoing work by Dr. David Olds and colleagues, there is a powerful body of evidence demonstrating the impact of the NFP model and a national quality assurance system that supports communities in replicating NFP with fidelity. But because, like all social service providers, we operate in a funding system that pays for activities, not results, we have grown by shoe-horning activities-based funding (e.g., government contracts that allow NFP to hire more nurses or to bill for more services provided to mothers) into our outcomes-oriented organization that carefully tracks the difference these services make in the lives of mothers and children. Pay for Success is attractive to us because it aligns funding with the focus on outcomes, which has always been core to NFP.
Our research partners were invaluable in the first step required to engage in Pay for Success work, delving into identifying three to five viable outcome metrics for Pay for Success from NFP’s larger body of demonstrated outcomes. We worked on model impacts, health economics, and replication data. We focused on those outcomes that achieve the most meaningful impacts for mothers and babies, produce the most significant savings for state governments, and can be efficiently measured in the Pay for Success measurement period.
The second step is the most complex: using existing and often imperfect data on baseline levels of impact to project the size of the effect in that community for the selected outcome metrics. Given the many “what ifs,” it is challenging to negotiate reasonable effect sizes that ultimately drive success payments. Some examples include the difficulty of reliably predicting future trends in community demographics and population health that can and will change the impact that any individual model, like NFP, can have. This work requires intensive analysis during deal construction but does not require a shift in how we deliver our services. In contrast, there is an intensified focus on enrollment as a result of Pay for Success.
One observation about our South Carolina deal may be useful. In South Carolina, a randomized controlled trial evaluation will be used to assess whether NFP can achieve the outcome metrics while also reducing costs through model modernization strategies, such as telemedicine and tailoring visit schedules to align more closely with the risk levels of each mother. This is requiring a deep dive to determine more precisely which program elements are most critical to outcomes achievement.
REFLECTIONS ON LESSONS WE LEARNED
Emphasizing that we are very much in the early stages of Pay for Success, and learning each day, we can safely share these thoughts. First, to make future projects more efficient to develop and launch, we are refining a national Pay for Success framework for NFP. This includes outcome metric options, but also a plan for which issues to tackle first in the deal construction process so we know if there is enough alignment in priorities to invest more time. Another element to this is the need for a contingency fund as part of the budget, since modeled programmatic costs will have some variation in practice.
Second, governments have surprisingly different appetites to pay for tangible vs. intangible outcomes, which is key to determining whether the math will work for the deal. For example, fewer child injuries requiring hospitalization reduces health care costs, a tangible savings to government. In contrast, improvements in quality of life from lower infant mortality rates produce intangible savings to society, measured in the value of a child’s life. It is also important which level of government benefits from outcomes. Federal participation in paying for outcomes would be very helpful for NFP, given that 62 percent of our government savings accrue to Medicaid.1
Third, there are key partners that have to be included. For providers, this work would be very difficult, especially at this early stage, to complete without an intermediary like Social Finance and the Harvard Kennedy School Government Performance Lab Innovation Fellows, but it is critical for providers to be at the table to reflect the needs of families served. Influential funders can play a vital role in encouraging state and community leaders to move a Pay for Success project forward in a responsible way and in creating a sense of urgency. Pay for Success deals seem to periodically run into a risk of crumbling under the weight of their complexity, and funders can be helpful in mobilizing toward the end goal.
Fourth, prepare for a roller-coaster ride. We have worked with three separate states on Pay for Success deals for the past two years and have launched one project (in South Carolina) so far. We often resort to labeling our Pay for Success projects as in the “zombie stage.” It can be hard to decipher when the projects are dead and when they will resurface. For example, in South Carolina, there were numerous points when the complexity of data sharing and evaluation or significant staffing changes resulted in long, painful delays. A major health crisis led to delays in another state. Pay for Success is unlikely to be the top priority for any government, but with persistence and lots of weekend and holiday work, the projects have come back to the priority work list and proceeded with great speed—only to completely stall a few weeks later.
Finally, enabling legislation for outcome payments is useful, but it also may be helpful for success payment funds to be captured in a place where they cannot be swept away in the future. This is why we needed Children’s Trust of South Carolina, where funds could not be re-appropriated by a new administration.
LOOKING FORWARD
Even for an organization celebrated for its data-driven culture and outcomes focus, NFP has had to put our performance management “on steroids” to meet the requirements of the South Carolina Pay for Success project. Pay for Success typically requires providers to develop and adhere to monthly enrollment schedules that are tied to project budgets. To stay on schedule and within budget, we now track enrollment daily rather than retroactively reviewing it each month or so. This requires dedicated staff and is part of what is transformative about Pay for Success. This will lead to more efficient use of public dollars by making sure that every available dollar is used to serve as many families as possible. But might it also perversely drive enrollment away from those with the greatest needs?
During the deal construction phase, it is important to recognize when to persevere and when to exercise the discipline to say no. It is important to recognize when Pay for Success is not the right tool to reach the people you are serving. For example, size matters. Small projects may lead to unacceptably high costs per person served overall due to fixed Pay for Success transactional costs, as we found in one potential deal so far. Or small projects may present challenges with statistical power, where we have too small of an intervention or a control group to be able to prove causality.
Pay for Success is very complex work that requires staying power. The headaches are worth it if we can simplify the process and allow successful programs to keep serving families. Sustainability could be a powerful provider incentive. If a provider delivers on the promise of Pay for Success and meets outcome metrics, government should make success payments and also commit to sustaining services going forward. That would be truly transformative.
If Pay for Success works at a larger scale, then NFP can realize a vision that we’ve shared with supporters for years: reaching every mom who needs us, sparking multi-generational change, and creating pathways out of poverty. Pay for Success holds the same promise for other participating nonprofits striving to reach their goals. In that way, despite challenges and drawbacks, Pay for Success holds the exciting promise of unleashing major, new sources of funding that can become models for a new wave of highly effective public-private partnerships.
1Ted R. Miller, “Projected Outcomes of Nurse-Family Partnership Home Visitation During 1996–2013, USA,” Prevention Science 16 (6) (2015): 765–777. Return cited here is based on investment calculator updated by Dr. Miller on March 7, 2017.