Performance-Based Contracting in New York City: Progressive Practice or Punitive Pursuit?
We made a conscious decision to serve families as long as they needed us. We did get cuts to our [New York City] contract for this. This is a concern because we’re dealing with real live human beings and [instead] we’re looking at numbers.
— NYC human services nonprofit staff member (NFF focus group, 2016)
Does the above scenario sound dire? Consider it in the larger context of nonprofits’ experience in recent years: In 2015, 84 percent of human services nonprofits (HSNs) surveyed in New York State reported an increase in demand for services over the previous year. At the same time, more than half (58 percent) reported being unable to meet that demand.1
While New York City HSNs continually struggle to meet the public’s growing need for their life-saving programs, they are often faced with an additional challenge: reconciling the needs of their clients with the mandates of city-generated, performance-based contracts, which tie payments to measurable outcomes.2 These contracts are part of a larger trend in procurement practices that began at the federal level as an effort to improve the efficiency and quality of service delivery. In recent years, performance-based contracting has become an increasingly preferred method for state and local government contracting as well.3 New York City has been a pioneer in implementing performance measurement standards at the local level, and in the context of New York City’s human services sector, performance-based contracts are intended to orient government and providers around the shared goal of offering the best outcomes for New Yorkers in need. Ironically, these contracts, which exist specifically to provide critical financial support to HSNs and incentivize outcomes-based practices, can also significantly interfere with that goal.
This chapter draws on data, input, and recommendations gathered from HSNs and other city stakeholders to examine this tension. In 2016, Nonprofit Finance Fund (NFF) disseminated two surveys about city performance-based contracts, and conducted a complementary set of four focus groups with HSNs, local government agencies, and researchers. Survey #1 is an in-depth survey that explored how HSNs’ organizational characteristics affect their ability to successfully compete for and manage performance-based contracts.4 Survey #2 solicited input from HSNs and city stakeholders on a set of 15 possible modifications to city performance-based contracts, generated from the previous survey’s findings and the focus groups.5 It is our hope that the feedback we received from our surveys and focus groups can serve not only as a bridge to effective collaboration within New York City, but also as a guide to other communities and sectors embarking on their own outcomes-based funding efforts.
DEFINING SUCCESS
Consider the two statements below and the deep divide in institutional perspectives that they reflect:
Government is best at setting outcomes, designing policy and overseeing and supervising performance.6
— Mark Hoover, former first deputy commissioner, NYC Human Resources Administration
Most benchmarks are developed [by the city] without input from providers, or in spite of it, and often do not reflect the complexity of the work nonprofits do, particularly with marginalized communities.
— HSN service provider comment (NFF survey, August 2016)
The most innovative element of a performance-based contract is its focus on defining programmatic outcomes, rather than programmatic process. This shift is intended to provide HSNs significant freedom in implementation, encouraging creativity and use of their applied expertise. However, because the city unilaterally defines the ultimate objectives of the performance-based contract — its outcomes — as well as the benchmarks and milestones that demonstrate progress toward the contract’s objective, disconnects like the one in the statements above can emerge. While performance-based contracts do provide substantial flexibility in process, the funder-mandated outcomes may result in significantly different opinions between governments and service providers regarding valid indicators of success.
By virtue of their role in administration and governance, city agencies may focus on cost-benefit analyses and budgeting constraints when developing service contracts. By contrast, HSNs are embedded in the communities that they serve and invested in meeting the specific needs of their individual clients. This divergence can be particularly pronounced and problematic in the context of performance-based contracts, in which the achievement of city agency-dictated outcomes determines HSNs’ reimbursement. Practices that appear cost-efficient and economical to a city agency may be at cross-purposes with an HSN’s goal of high-quality, client-driven services. This is especially true when the nonprofit serves clients who are members of marginalized communities. These service providers are less likely to have had a platform to voice their community’s needs, challenges, and experiences to their government funders.7
An additional consideration that should factor into performance-based contract benchmarks is accommodating the potentially profound effects that external factors wield on HSN program implementation. Because these factors are often both unexpected and beyond the control of HSNs, the performance-based contract structure is useful only insofar as it is able to adapt and respond to fluctuations in the need and context for services. As an HSN professional described, “We recently had to write an appeal to [our funding agency] because…performance-based contracts don’t take into account external variables that affect our ability to manage these contracts” (NFF focus group, 2016).
From previous conversations with HSNs, we have heard that these externalities can include shifts in the political climate or unexpected events, such as Hurricane Sandy in 2012 and the sudden disintegration of the Federation Employment & Guidance Service (FEGS) in 2015. Our survey findings confirm the importance of giving service provider organizations a larger role in determining appropriate outcomes and milestones for performance-based contracts, not only at the outset of the contract but also throughout the duration of the contracting relationship.
Recommendation: We recommend that government contract managers collaborate quite closely with HSNs who are expert in the communities that they serve, especially when those clients are members of groups who may have significant historical reason to distrust traditional authority figures like government officials. This collaboration should not only inform the initial content and structure of performance-based contracts, but also should remain part of the process of contract fulfillment, with contract managers and HSNs continuing to be in close communication.
MEASURING SUCCESS
In addition to defining success, measuring success is also an integral element of engaging in performance-based contracts. For an HSN to measure its performance, it must have the capacity for data collection and evaluation. It is rare for any type of government contract to cover a provider’s full cost of services, but performance-based contracts put providers in a particular double-bind: They require HSNs to collect data and report on performance but typically fail to fund these resource-intensive activities.
This unfunded element of performance-based contracts creates bias in the awarding of contracts. Specifically, organizations that are large and well-funded enough to invest in data collection and evaluation capacities are able to pay for the cost of demonstrating performance and outcomes, rendering organizations without this capacity or budget flexibility — often smaller, newer, or possibly serving especially vulnerable and high-poverty communities — unable to compete for performance-based contracts.
These smaller organizations’ limited resources are likely to be consumed by client service delivery, without available funding for specialized administrative roles or data management systems. Indeed, data from Survey #1 demonstrated that larger organizations (as defined by budget size) were more likely to have made successful bids for performance-based contracts, arguably because they have staff and resources allocated toward packaging successful applications.
On the other hand, findings from Survey #1 revealed an exception to this trend: Organizations with “specialized staff” (e.g., staff roles that include data management, strategy development, or grant writing) were more likely to have successfully bid for a performance-based contract than organizations without specialized staff, regardless of organization size.
Recommendation: A critical component of ensuring that performance-based contracts can function effectively is providing support for all of their activities. In this case, that includes not only providing funding for client-focused services, but also adequate financial support for HSNs to engage in data collection and report on the required client outcomes. We recommend that cities adopt performance-based contracting practices that compensate HSNs for data collection and reporting activities. This will ensure that highly qualified service providers are not excluded because they are too small or not well-resourced enough to have the budgetary flexibility to absorb the costs associated with data collection. This is an especially important consideration if communities are invested in ensuring that contracts are awarded to HSNs who are truly providing the highest quality services to clients, not only those who have the budget flexibility to cover the costs of outcomes management.
HSNs feel the strain of having to bear the cost of collecting and providing the outcomes data required of performance-based contracts. In Survey #2, HSNs expressed a uniform plea that performance-based contracts cover both the complete cost of providing high-quality service delivery as well as indirect costs. These include costs associated with running high-quality organizations, such as administration, facilities, and outcomes measurement. Interestingly — and perhaps not surprisingly — in their survey responses, city stakeholders and contract managers assigned notably less importance to funding modifications that addressed indirect costs. This highlights a disconnect between HSNs and city stakeholders in their perspectives regarding funding practices, and therefore an opportunity for city agencies to hear why covering full costs, including outcomes measurement, is key for HSNs’ success.8
COLLABORATING FOR SUCCESS
We posted a concept paper for feedback and only one person responded. And that person was in Thailand!
— NYC agency staff member (NFF focus group, 2016)
Before publishing a formal request for proposal to solicit HSNs’ applications for service contracts, the city distributes a “concept paper” that invites feedback on the proposed intervention and service model. As expressed in the quote above, from the city’s perspective, opportunities for collaboration on contract development do exist, but HSNs do not engage them. This stance appeared at odds with the one held by HSNs, who consistently reported a strong desire for a much greater role in the process of creating contracts’ structure and content. However, a closer examination of the data and input from both groups reveals a more complex dynamic.
Surveys #1 and #2 yielded consistent feedback from HSNs. For example, in Survey #1, when asked to select the single practice that would be most helpful in building capacity to meet the demands of performance-based contracts, the most frequently selected option was, “Give service provider organizations a larger role in defining the structural elements of performance-based contracts” (40 percent). The second most frequently selected option was, “Give service provider organizations a larger role in determining appropriate outcomes and milestones/benchmarks for performance-based contracts” (26 percent).9 When pressed to reconcile this strong desire to participate in contract creation with their unresponsiveness to city-generated concept papers, HSN focus group participants revealed dissatisfaction and disengagement with the current concept paper input process. They proposed the following modifications to how the city gathers HSN feedback on contracts:
- Develop a system in which concept papers are published in a central location for a period of time that is predictable and adequate for submitting feedback. This would enable HSNs to be aware of all concept papers relevant to the communities they serve and have sufficient time to respond and contribute their content expertise.
- Expand the scope of concept papers. Many are limited to somewhat theoretical content and omit elements of the contract, such as progress milestones or payment structure.
- Improve the feedback review process for concept papers. Currently, HSNs’ input and suggestions are not acknowledged in any way. The city should acknowledge feedback and outline which recommendations were included and excluded in the final contract, as well as the basis for these decisions. Such feedback would be good progress toward facilitating a dialogue between the city and its human services sector.
Interestingly, when we reported these recommendations back to city representatives, we learned that strict procurement standards — created with the intention of preventing bias or conflicts of interest — can, in fact, inhibit interaction and a meaningful feedback loop between city agencies and HSNs. As one city representative expressed during a focus group: “Procurement considerations that were set up to create fairness and openness are prohibiting real and meaningful conversations” (NFF focus group, 2016). This concern was echoed in stakeholders’ responses to Survey #2. From a list of 15 possible refinements to performance-based contracts, “Adjust procurement rules so that city agencies may solicit input from nonprofits without concerns of appearing biased” scored the highest among city stakeholders (excluding HSNs), with a median score of 10 out of 10 (i.e., “extremely helpful”) and an average score of 8.10
Recommendation: We recommend that government officials consider modifying procurement policies, adding systematic opportunities for significant and meaningful interaction between contract managers and HSNs, while still ensuring neutrality in the processes of shaping performance-based contract outcomes. In addition, we suggest that communities interested in employing performance-based contracts create a single online clearinghouse for concept papers, and ensure that they are available for a consistent amount of time, such that HSNs are able to review and thoughtfully respond to them. This clearinghouse should also include a function for publicly listing feedback from HSNs on concept papers, and specifying decisions on whether to incorporate that feedback.
CONCLUSION
The objective of this research was to develop an understanding of how performance-based contracts affect New York City HSNs, and to use those insights to make a series of practical recommendations to cities and communities across the country. Two surveys and several focus groups later, we feel that we’ve gained insight into the challenges that HSNs face when they engage performance-based contracts; we’ve also learned about some of the challenges faced by city agencies that make it difficult to resolve those issues within the complex and fluctuating ecosystem of New York City. We hope this will spark discussion and provide some useful input as city government and HSNs work together to achieve strong outcomes for New Yorkers and communities beyond.
Many thanks to the Human Services Council (HSC) and Jina Paik of Nonprofit Finance Fund for their collaboration, insight, and many excellent jokes during this research. In addition, we would like to thank Professor L.L. Martin of the University of Central Florida for lending us his resources and sharing his expertise on performance-based contracting. Finally, thanks to Professor Christa Altenstetter of the CUNY Graduate Center for her guidance, mentorship, and intellect on this project among many over the years.
1 Nonprofit Finance Fund, “State of the Nonprofit Sector Survey” (2015), available by request.
2 There is no universal definition of performance-based contracts. We used the following definition, which is a synthesis of a series of working definitions written by various government agencies and associations: “A performance-based contract can be defined as one that focuses on the outputs, quality, and outcomes of a service provision and may tie at least a portion of a contractor’s payment as well as any contract extension or renewal to their achievement,” as defined in Lawrence L. Martin, “Performance Contracting: Extending Performance Measurement to Another Level,” Public Administration Times 22(1) (1999): 1 & 2.
3 Lawrence L. Martin, “Performance-Based Contracting,” Government Contracting: A Public Solutions Handbook, ed. Robert A. Shick (New York: Routledge, 2016).
4 A total of 53 nonprofit employees participated in Survey #1 in April and May of 2016. Survey #2, disseminated in August 2016 based on findings from Survey #1, received a total of 45 responses. These employees represent organizations throughout New York City that varied in terms of size, budget, and current and anticipated use of performance-based contracts.
5 A total of 45 employees from nonprofits, city agencies, funding agencies, and advocacy organizations completed Survey #2.
6 Bryna M. Sanger, The Welfare Marketplace: Privatization and Welfare Reform, Brookings Institution Press (2003).
7 For an excellent exploration of how certain communities and beneficiaries have been classified as more deserving of investment and resources than others, see Anne L. Schneider, Helen Ingram, and Peter Deleon, “Democratic Policy Design: Social Construction of Target Populations,” Theories of the Policy Process (3rd Edition, 2014: 105–149).
8 The survey options “Incorporate full cost of services into payment tiers” and “Incorporate indirect cost of services into payment tiers” received average scores above 9 on a 10-point scale (9.5 and 9.3 respectively, with 10 being “extremely helpful”). Also notable, not a single HSN respondent assigned either option a score below 6. This demonstrates that the average level of support for these modifications is not only exceptionally high, it is also uniform across the entire group of HSNs. By contrast, city stakeholders and contract managers assigned notably lower scores to these two possible modifications, yielding average scores of 6.8 and 6.2, respectively.
9 In Survey #2, HSNs were presented with a list of 15 possible modifications to performance-based contracts — including the same two options so heavily favored in Survey #1. Their responses yielded an average score of greater than 9 on a 10-point scale (9.1 and 9.3, respectively, with 10 being “extremely helpful”).
10 City stakeholders include employees of city agencies, funding agencies, and advocacy organizations.