Better Outcomes for the Nonprofit Sector
How do you know? Sixteen years ago, as a new nonprofit executive director, I asked that question repeatedly about our impact, efficiency, quality, financial stability, reputation, and so on. We didn’t know then how well we performed by any of these measures; today, we do.
While visiting programs in those early days, I’d ask staff how they knew if we, the organization, and they, the staff, were good at what we did. It wasn’t a trick question. As the new guy, I was eager to find the secret to our success and the areas in which we needed to improve. The response was always a moving story of a client and the details of how this program and that staff helped him or her achieve sobriety, stabilize mental health, find a job, and eventually obtain housing. But when I asked how often that occurred, what contributed to the result, and what it would take to increase the rate of these incredible, positive outcomes, I might as well have been speaking a foreign language. Nobody knew; all they could tell me was the story of their client.
The overwhelmingly positive energy from staff passionate about their work and the people they serve might have been sufficient for some (particularly in a sector where the workforce often feels insufficiently supported and morale can be low). But as powerful as that impression was, what I found even more striking was the absence of any sense of the systemic impact of their work, of the outcomes they achieved, and of how, with all their passion, they could — we could — do more good, more often, for more people.
Further, like many nonprofits, we were facing huge financial challenges. I knew that if we could grow by replicating the work we did well, the economies of scale (combined with other efforts) could improve our financial condition, while simultaneously advancing our mission and enabling us to help more people. But to make the case for growth to our funders, I needed to demonstrate impact.
Our organization was founded nearly 50 years ago by recovering alcoholics living in New York City’s Bowery flophouses. They were “the homeless” of their era, before the phrase was coined. Today, we stay true to their belief that no matter how far someone has fallen, he or she can climb back and achieve if given an opportunity and support. We provide holistically integrated services and housing to make that possible for anyone willing to try, no matter what came before. More simply, we offer a hand up instead of a handout and provide the opportunity for our clients to achieve their potential.
As I listened to the individual stories in those early days, I was humbled by the challenges our clients overcame. Yet we cannot run, sustain, and grow our organizations by relying solely on emotion. We need proof of our impact and evidence of the key variables that contribute to the results. So we created a system for measuring performance and management, assessing our work, and instilling a culture that values inquiry and encourages adaptive management.
This led us on a journey of inquiry and analysis, of acquiring knowledge and learning from it — a journey that continues today. There was fear when we began: Would the pursuit of data change our mission or cause us to focus our energy only on clients likely to succeed? Would people lose their jobs if their outcomes weren’t good enough?
What I said then, and say today, is that we must strive to manage with data, not to it. Am I excited when a program achieves a good outcome? Absolutely. Am I concerned when it does not? Of course. But I am much more interested in why. Data are not an outcome but an accounting instrument. Like staff, facilities, and dollars, they are a tool and resource with which we manage. How we use these resources determines how we achieve outcomes. Sustaining and replicating success is unlikely if we don’t understand how it happened. Similarly, if we understand why some results aren’t what we expected, we have the ability to turn them around.
Here’s an example. We assess the performance of our homeless shelters largely by their success at helping people find and retain housing. We measure how many people get housed (overall and as a percentage of those served), how quickly that happens, and whether those who leave us stay housed. A few years ago, we noticed that performance was declining. We were initially patient; it was important to know whether this was a trend. Because a key staff member was on family leave, it seemed reasonable to attribute the drop in performance to her absence. The trend continued when our staffer returned, so we dug deeper. Might the performance of our complementary workforce development program be impacting the shelter’s outcomes? The data said no; in fact, our clients were entering the labor force at ever greater average wages and were sustaining employment at consistently high rates. Further analysis revealed the source of the decline: Our working clients had fewer housing options, a result of the well-documented elimination of nearly all rental subsidies for housing, compared with prior years.1 As a result, it was taking them longer to save enough to move out on their own, they were moving to less secure situations (renting rooms without tenancy rights; sharing costs with roommates; renting apartments they could not afford), and therefore were more at risk of losing their housing and returning to the shelter. Thus, we realized that our shelter’s performance was declining despite, not because of, our efforts.
We set out to find a new way to improve our ability to meet our clients’ needs for affordable housing that would also decrease lengths of shelter stays and reduce the likelihood that they would return to the shelter. We realized that in order to more positively impact our clients’ lives, and our performance, we would need to focus on producing low-income housing. So we developed a financing model to build housing they could afford, and now we are building that housing: 135 apartments in the Bronx with rents that will be affordable to our clients (studios for less than $500 per month for people earning less than $20,000 per year). It’s half of an innovative financing model that also includes the development of a new homeless shelter. It leverages revenue from the shelter — what would otherwise be the profit we generate for a private developer when we lease our shelters — and repurposes these dollars to provide the operating subsidy needed to make the housing affordable. It’s a model that can be replicated to develop housing and strengthen nonprofits’ financial condition.
If we had not been outcome-oriented, we might never have realized the challenge and devised a solution. That said, it’s important to be cautious when using data to evaluate performance. The decline in our ability to move clients to housing had occurred despite our best efforts and was attributable to external factors. Yet, if we had been funded by a typical performance-based contract, we would have lost income and likely not been able to invest in analysis and innovation. This is because current performance-based contracts have two major flaws: 1) they set performance targets and evaluate them on an absolute basis, without considering context, and 2) they provide reduced payment when targets aren’t hit. There is a better, more productive way to configure outcomes-based contracts, one that would ultimately drive better results for the clients of homeless services organizations. What could that look like?
Nonprofits and those who fund them should care about and measure performance; more important, they should identify and account for what drives the data. Rather than setting a target of a specific number of units, funders could say: I’ll pay you for putting as many people as possible into permanent housing and keeping them there. They pay for results on a per person basis, and can consider relative performance across organizations. This allows them to ask: For every 100 clients served, how many was organization A able to successfully transition to permanent housing? How about organizations B, C and D? That in turn creates a level playing field and doesn’t penalize organizations for external factors outside of their control (e.g., skyrocketing real estate prices, government policy changes).
Secondly, we must stop the practice of paying reduced amounts as the penalty for not hitting performance targets. Many organizations have operated on shoestring budgets for decades; reducing their payments and asking them to do more with less money is a recipe for failure. Instead, funders should pay the full amount the first time an organization underperforms, and tell them they need to improve. Expectations should be clear and direct, and the nonprofit’s leadership should be given a reasonable timeline to make improvements. If results don’t improve in that reasonable timeline, all of their funding should go to other organizations that are better achieving results for clients. We have all become accustomed to getting paid for effort, not outcomes. And while the sector’s effort is laudable, perhaps a little competition is healthy for us. And ultimately healthy for the clients we all serve.
This approach to nonprofit management is imperative at a time when skepticism is growing about the integrity of all institutions. Nonprofits cannot expect to be, or ask to be, exempt from this scrutiny because of their mission’s worthiness. Therefore, it is not sufficient for us to take comfort in leading our own high-performing organizations; those of us who value this accountability have a responsibility both to assist our peers who seek to follow suit, and to call out those who do not. The further transformation of our sector toward one that values and speaks thoughtfully about outcomes, and not simply effort, will fundamentally improve the lives of those we serve, consistent with our missions. If we fail to do so, we should not be surprised by nor bemoan those who question the value of our sector or seek to disinvest from it.
I’m proud of what we do and how well we do it. I know we have room to improve, and I know where. And I still ask that same question: How do you know? Only now, we have a system to generate the answers.
1 Matt Chaban, “Tenants and Landlords Fret as Sequester Cuts Hit Section 8 Subsidies,” Crain’s New York Business (2013), available at http://www.crainsnewyork.com/article/20130609/REAL_ESTATE/306099969/tenants-and-landlords-fret-as-sequester-cuts-hit-section-8-subsidies.