Dear Vu,
I just read your blog (as I often do because it’s good stuff that everyone should read) and as with your post a few weeks ago about nonprofits not acting enough like businesses, I felt the need to respond to this one.
First of all, I should say that I agree wholeheartedly with the points you make about the difficulty of defining, achieving, and measuring nonprofit outcomes. Nonprofits do not operate on a strictly dollar-for-dollar basis where outputs are instantly converted to profits. Our return on investment is measured not products sold or dollars earned but in social change over years. Producing outcomes takes dedication, patience, and yes, sophisticated measures of evaluation that donors don’t often fund. I like that you point out the non-linear nature of the work and the importance of process, because we need to remember that things are not always straightforward and that’s okay.
That said, I’m surprised at how insulted you are at the suggestion that nonprofits do not have clear outcomes. Now, granted, whoever wrote that comment about outcomes and ROI may have a certain meaning about outcomes that’s more about the very nature of nonprofit work. But in a more literal sense, the fact is that many nonprofits do not, in fact, have clear outcomes.
I viewed your organization’s evaluation map and I was thoroughly impressed. But you have to understand that many, many organizations do not have such thoughtful or sophisticated logic models for their organizations. Heck, I’ve worked with organizations that don’t even have logic models for any of their programs, let alone ones that include inputs and tie program outputs to short-term and long-term outcomes. Are there organizations that are as on top of things as your organization is? Absolutely. But throughout my career – first in nonprofit organizations, then in philanthropy, and more recently as a nonprofit consultant – I can say that your evaluation map is indeed the exception, not the norm. I would love to be proven wrong – to learn that the vast majority of nonprofits have thoughtful logic models for their organizations – but my experience has been that too many do not have them.
The map aside, let me tell you why I don’t think nonprofits have clear outcomes: they don’t know how to define an outcome. Sure, they may think they do, but time and time again I see outputs instead of outcomes. Yes, outputs are easier to define and measure, absolutely they are. But that doesn’t make them outcomes. Even in your own evaluation map, which I think is top-notch, there are outputs listed as outcomes (e.g. gaining skills and participating in advocacy efforts under short-term outcomes, more clients served and increased number of leaders in intermediate-term outcomes). From my perspective, you may call these outcomes, but they aren’t really. Results? Yes. Change? Sure. But outcomes? Not quite. It doesn’t detract from their value or importance, but again, I would say they aren’t clear outcomes.
And yes, some outcome statements are indeed fuzzy. I’m not talking about the “warm and fuzzy” kind of outcome (though there are plenty of those in the nonprofit world – and I often like them), but fuzzy in terms of their definition and clarity. This is usually because they’re worded in a way that leaves it unclear as to what the outcome is or how one would know if it was achieved. For instance, on your own evaluation map, Outcome 6 is “Increase in meaningful collaborative activities.” What do you mean by “meaningful” in this context? What exactly are collaborative activities? And when you say increase, does that mean that one more than the current number is sufficient to claim success? Yes, I’m being excessively critical, but I’m also trying to point out why outcomes are often seen as unclear or fuzzy. Even when you have a true outcome (as opposed to an output), if you don’t state it in an articulate way, it will open you up to the criticisms to which you take such great offense.
I should mention that I work with lots of organizations on this exact problem – clarity and definition of outcomes – because I too hate the criticisms. I am adamant that nonprofit work is worth the investment, even if it isn’t a financial return that you get. But I also hear the criticisms and try to learn from them, and the truth is that nonprofits can do better at defining and articulating their outcomes.
So what can nonprofits do to improve their outcome statements and reduce some of the criticisms (given that you won’t please the people who just want everything measured in dollars of ROI)? Here are some basic steps to take:
- Know what an outcome is. Outcomes are not the same as outputs, but rather are a change in the behavior of people, institutions, or a system. This is the real, lasting change that you’re after with your work. It’s what we call “impact.”
- Frame your outcomes as SMART goals. Yeah, yeah, I know we’re all sick of hearing about SMART objectives, but if you can state your outcomes in specific, measurable, and even time-bound ways, it gives them greater definition and clarity. Saying things will be “better” or “more effective” doesn’t really give much definition to an outcome.
- Think in concrete terms. I’ve written before about improving your communications by cutting the crap, but this is also true for outcome statements. If you don’t want people to think your outcomes are fuzzy or fluffy, then make sure they are worded in a way that paints a very concrete picture of what you mean. Don’t leave people with questions about what exactly your outcome is. Be concrete. Be direct. Cut the crap.
Again, I don’t disagree with you on the challenges of defining, achieving, and measuring outcomes in the nonprofit world. And I know that you’re responding to a slightly different criticism that’s more about the nature of nonprofit results. But I wanted to present my own tough love perspective, which is to say that nonprofits need to do a better job of defining and articulating their outcome statements. I’m not saying it’s always easy, but it will reap rewards: not only will it reduce such criticisms, but it should create a more compelling case for support and it should give organizations more clarity and focus in their work.
Thanks for always putting forth such great thought-provoking pieces, and an early congratulations on your forthcoming addition to the family!
Eric
Proaireton Consulting 