Narrowing Focus Was the Key to Transforming the Lives of High-Risk Youth in Massachusetts
At Roca, we know from experience that good intentions don’t always produce good results.
In the early days of the 1990s and early 2000s, Roca was a loosely defined youth development organization that wanted to help everyone we could. Just over the Mystic River from Boston, Roca’s big white building stood out as a beacon of hope in the otherwise economically and socially depressed community of Chelsea.
In this gateway city, a harbor for the region’s poor and largely disenfranchised minorities, youths as young as eight and all the way up to 25 struggled to leave the streets and avoid gang life; to make it through school without becoming pregnant or addicted to drugs; to deal with complicated immigration statuses; to protect themselves from domestic or peer violence; to just make it to adulthood, relatively unscathed.
But no matter your problem, Roca (which means “rock” in Spanish) was there to help solve it — or at least we’d try. A de facto multiservice and multicultural community center at that time, we didn’t just provide street outreach to high-risk young people, as we continue to today. We also ran dance classes, summer leadership programs, social ventures, afterschool tutoring, fitness classes, a music studio, and sports leagues.
At times, the one focus we had — youth — felt too narrow, even selfish. So we also ran a weekly food pantry for the homeless. We volunteered our space for local church groups to hold services on the weekends. We provided adult basic education and literacy programs. We ran immigration clinics so that the newest members of our community would know their rights.
Like any well-intentioned soul inspired by the John Wesley quote, “Do all the good you can, by all the means you can, in all the ways you can,” Roca strove to be all things for all people. Our good intentions knew no bounds.
And in the dawn of positive youth development as the social service “gold standard” for building on the strengths and skills of disconnected young people, our energetic, scrappy, and relentlessly optimistic staff recruited youth of all ages and risk levels into positive, strengths-affirming activities of all kinds. We didn’t discriminate as to which youth we thought we could effectively serve, nor did we carefully select the services we would use to try to help them. We provided a long menu of engagement activities for youth to choose from and crossed our fingers that we were helping them change their lives, get out of poverty, and stay out of harm’s way.
But when we began to realize that engagement alone didn’t seem to be deterring them from negative behaviors and situations, we were forced to reexamine our approach. Why, we wondered, was youth violence not dropping in the community? Why were the lower-risk kids suddenly joining the most notorious local gangs? And why were the clear indicators of community poverty — rates of teen pregnancy, crime, and school failure — still on the rise?
The problem was that our model wasn’t just unsustainable — it was potentially dangerous. Given the realities of the Chelsea streets and persistent youth violence, we feared that despite our own best intentions, we were running a self-esteem program for gang members: Young people who were winning Roca basketball tournaments by day were shooting at people by night.
Roca’s hard-fought journey to becoming a high-performing, effective nonprofit began when we finally asked ourselves the critical, dual question, “Are we helping young people change their lives, and how do we know?”
After realizing that, despite great dedication and hard work, Roca was not helping young people improve their lives to a significant degree, we knew it was time to take stock and rethink what we were doing entirely.
Our young people, many of whom had grown up at Roca, deserved and needed the organization to get better at its mission so that they could, in fact, move out of harm’s way and toward economic independence. To help young people change, Roca had to change first.
Over several formative years for the organization, Roca engaged in a systemic cycle of research, design, action, and use of data for continuous improvement of our model in order to deliver an intervention worthy of the young people we serve. After a series of theory-of-change processes, we narrowed our target population to the highest-risk young people, aged 17 to 24 — those whom most programs give up on and whom the data showed that Roca was actually successful at helping.
Critical to the model we have today, we studied, adapted, and implemented evidence-based practices from behavioral health, criminal justice, community corrections and reentry, medical and mental health, and workforce development to inform and direct our approach to behavior change and skill-building for high-risk young people. Meanwhile, we deepened our capacity to collect, analyze, and use data regularly as a tool to inform our strategic and operational goals. Finally, we clarified Roca’s intervention model as a single-service intervention for high-risk youth. We provide two years of intensive services (case management, life skills, education, and employment programming) and two years of less intensive services, replete with multiple opportunities along the way for young people to work on long-term behavior change through intensive relationships with youth workers.
In other words, we closed the community center and opened up a data-driven, targeted program worthy of serving young people in crisis — those who were in and out of jail, in gangs, had dropped out of school, and were not ready, willing, or able to reverse these destructive patterns without help. We forced ourselves to stop working with the groups we weren’t helping, as painful as this was. We designed a model specifically for the high-risk young people whom we knew Roca had the means and the experience to help. We put start and end times around how long we would work with young people intensively (up to two years — any longer, and the data showed that youth began to regress). And we built programming to support the changes we wanted young people to learn to make: to drop their guns, stay out of jail, and learn how to go to work. Meanwhile, we built an internal culture of performance-based management to hold ourselves accountable to each other, our partners, and the young people we serve.
That meant that all of us — staff, management, partners, funders — had to learn what it means to focus on outcomes and on doing things that matter. We had to build the staff’s capacity to work with our data system, read reports, and discuss what could be done differently. We had to explain, over and over, that numbers are not just numbers; they are real people, and they reflect a reality that changes (or not) for each of the young people we serve. We had to maintain our ability to have hard conversations, using peacemaking circles and other methods we have learned over the years, because when you strive to focus on what works and let go of what doesn’t, hard conversations will have to happen.
We also had to develop a new type of conversation with our funders. Now that we were able to generate detailed reports about our intervention model, we also had to be accountable for what the reports showed. We learned that if we can be painfully honest with our young people and with ourselves, we can also be as honest with our funders. We learned that you can tell a funder that you used its funding to try something you thought would work, tried it, tested it, found that it doesn’t work, discontinued it, and now are doing something else — all while maintaining and even increasing the funding from this funder. In fact, focusing on meaningful outcomes grew our donor base and brought us and our funders closer. Funders, too, want to do things that matter.
And we knew that if, after all was said and done, we still weren’t effectively helping young people, then we might as well close our doors. It isn’t enough to just “try” to help our young people change their lives. The stakes are too high. If we don’t do our jobs well, the young people we serve remain impoverished, they continue to get arrested, and most tragically, some commit violence or die from it.
This terrible reality is Roca’s call to action, and it’s why our organization compulsively uses data as a companion to experience. It’s why we’ve spent the better part of the past decade building a model that focuses on results first.
We track the performance of both young people and staff so that we can best understand the level of effort that is necessary to do this work well. We use data daily, weekly, monthly, quarterly, and annually to review how we’re doing, improve the model, and compare it with other effective models. Be it a strategic conversation between a youth worker and a participant, or a participant’s attendance at a pre-vocational training class, every component of Roca’s model is designed to drive young people toward positive outcomes based on sustainable behavior change and skills acquisition.
Roca is proud to be one of the fortunate, few organizations that is able to track whether it’s doing meaningful, measurable good on a regular and predictable basis. In fact, the data point to a robust model that is improving over time: In 2012, 90 percent of participants in the follow-up and retention phase of Roca’s model had no new arrests, increasing to 93 percent in 2015; in 2012, 79 percent of participants retained employment, increasing to 92 percent in 2015.
This continual use of data has utterly transformed Roca as an organization, and the accompanying success rates of our highest-risk young people have set the stage for bringing Roca’s impact to scale. In the past five years, we have become the primary service provider for our state’s juvenile justice Pay for Success program, have completed two in-state program replications in Boston and Springfield, have opened a satellite office in Lynn, and are exploring national options for expansion. Today, Roca serves over 900 high-risk young people per year from over 21 communities in Massachusetts. Embarking on the journey of studying ourselves, collecting data, and driving toward meaningful outcomes was one of the most effective steps we could have taken as a youth development organization for our young people.
With governments across the country looking for more efficient, less costly ways to address the most intractable social problems, such as poverty, homelessness, and mass incarceration, performance-based funding mechanisms open the door for highly successful, results-driven organizations to differentiate themselves from the nonprofit pack. At the same time, governments are encouraged to be selective in the organizations they fund, using performance data and outcomes as the primary tool for judging a program’s effectiveness. And in the face of these elevated standards, we all become better.