Category: logic model

communications, logic model, messaging, nonprofit sector, strategy

Here’s why your outcomes aren’t clear

Dear Vu,

I just read your blog (as I often do because it’s good stuff that everyone should read) and as with your post a few weeks ago about nonprofits not acting enough like businesses, I felt the need to respond to this one.

First of all, I should say that I agree wholeheartedly with the points you make about the difficulty of defining, achieving, and measuring nonprofit outcomes. Nonprofits do not operate on a strictly dollar-for-dollar basis where outputs are instantly converted to profits. Our return on investment is measured not products sold or dollars earned but in social change over years. Producing outcomes takes dedication, patience, and yes, sophisticated measures of evaluation that donors don’t often fund. I like that you point out the non-linear nature of the work and the importance of process, because we need to remember that things are not always straightforward and that’s okay.

That said, I’m surprised at how insulted you are at the suggestion that nonprofits do not have clear outcomes. Now, granted, whoever wrote that comment about outcomes and ROI may have a certain meaning about outcomes that’s more about the very nature of nonprofit work. But in a more literal sense, the fact is that many nonprofits do not, in fact, have clear outcomes.

I viewed your organization’s evaluation map and I was thoroughly impressed. But you have to understand that many, many organizations do not have such thoughtful or sophisticated logic models for their organizations. Heck, I’ve worked with organizations that don’t even have logic models for any of their programs, let alone ones that include inputs and tie program outputs to short-term and long-term outcomes. Are there organizations that are as on top of things as your organization is? Absolutely. But throughout my career – first in nonprofit organizations, then in philanthropy, and more recently as a nonprofit consultant – I can say that your evaluation map is indeed the exception, not the norm. I would love to be proven wrong – to learn that the vast majority of nonprofits have thoughtful logic models for their organizations – but my experience has been that too many do not have them.

The map aside, let me tell you why I don’t think nonprofits have clear outcomes: they don’t know how to define an outcome. Sure, they may think they do, but time and time again I see outputs instead of outcomes. Yes, outputs are easier to define and measure, absolutely they are. But that doesn’t make them outcomes. Even in your own evaluation map, which I think is top-notch, there are outputs listed as outcomes (e.g. gaining skills and participating in advocacy efforts under short-term outcomes, more clients served and increased number of leaders in intermediate-term outcomes). From my perspective, you may call these outcomes, but they aren’t really. Results? Yes. Change? Sure. But outcomes? Not quite. It doesn’t detract from their value or importance, but again, I would say they aren’t clear outcomes.

And yes, some outcome statements are indeed fuzzy. I’m not talking about the “warm and fuzzy” kind of outcome (though there are plenty of those in the nonprofit world – and I often like them), but fuzzy in terms of their definition and clarity. This is usually because they’re worded in a way that leaves it unclear as to what the outcome is or how one would know if it was achieved. For instance, on your own evaluation map, Outcome 6 is “Increase in meaningful collaborative activities.” What do you mean by “meaningful” in this context? What exactly are collaborative activities? And when you say increase, does that mean that one more than the current number is sufficient to claim success? Yes, I’m being excessively critical, but I’m also trying to point out why outcomes are often seen as unclear or fuzzy. Even when you have a true outcome (as opposed to an output), if you don’t state it in an articulate way, it will open you up to the criticisms to which you take such great offense.

I should mention that I work with lots of organizations on this exact problem – clarity and definition of outcomes – because I too hate the criticisms. I am adamant that nonprofit work is worth the investment, even if it isn’t a financial return that you get. But I also hear the criticisms and try to learn from them, and the truth is that nonprofits can do better at defining and articulating their outcomes.

So what can nonprofits do to improve their outcome statements and reduce some of the criticisms (given that you won’t please the people who just want everything measured in dollars of ROI)? Here are some basic steps to take:

  • Know what an outcome is. Outcomes are not the same as outputs, but rather are a change in the behavior of people, institutions, or a system. This is the real, lasting change that you’re after with your work. It’s what we call “impact.”
  • Frame your outcomes as SMART goals. Yeah, yeah, I know we’re all sick of hearing about SMART objectives, but if you can state your outcomes in specific, measurable, and even time-bound ways, it gives them greater definition and clarity. Saying things will be “better” or “more effective” doesn’t really give much definition to an outcome.
  • Think in concrete terms. I’ve written before about improving your communications by cutting the crap, but this is also true for outcome statements. If you don’t want people to think your outcomes are fuzzy or fluffy, then make sure they are worded in a way that paints a very concrete picture of what you mean. Don’t leave people with questions about what exactly your outcome is. Be concrete. Be direct. Cut the crap.

Again, I don’t disagree with you on the challenges of defining, achieving, and measuring outcomes in the nonprofit world. And I know that you’re responding to a slightly different criticism that’s more about the nature of nonprofit results. But I wanted to present my own tough love perspective, which is to say that nonprofits need to do a better job of defining and articulating their outcome statements. I’m not saying it’s always easy, but it will reap rewards: not only will it reduce such criticisms, but it should create a more compelling case for support and it should give organizations more clarity and focus in their work.

Thanks for always putting forth such great thought-provoking pieces, and an early congratulations on your forthcoming addition to the family!

Eric

Advertisement
leadership, logic model, planning, rationale, strategy, theory of change

6 lessons nonprofits can learn from the Avengers

It’s no secret that I’m a fan of superheroes. And why wouldn’t I be? They’re all about people doing extraordinary things to tackle problems for the greater good.

This year saw the second team up of Earth’s Mightiest Heroes in Marvel’s Avengers: Age of Ultron, bringing Iron Man, Captain America, Thor, Black Widow, Hulk, Hawkeye, and others together to take down a global threat. While the movie had its ups and downs, I realized that nonprofits can learn a lot from this group of do-gooders. After all, they too face seemingly insurmountable problems and have to overcome internal and external obstacles to achieve their goals.

So here are six lessons nonprofits can learn from the Avengers:

  1. You can’t do it alone. The Avengers are a group of superheroes who do plenty of good on their own. They each have their enemies and problems that they tackle in their own corners of the world (and universe). So what would force these different heroes to come together? A problem that none of them could overcome by themselves. Yes, they had differences to work through (see below) but they knew they needed each other because none of them could do it alone. Nonprofits are often like Iron Man or Thor, thinking they are mighty enough to do it all without help. Others are like Captain America, rallying others to work together, some are like Black Widow or Hawkeye, ready to work with others as needed, and others are like Hulk, afraid of working with others at all. The bottom line is that if nonprofits want to tackle big problems at scale – solving systemic, root causes – they need to come together. This is how the buzzword “collective action” came about: the idea that we need to work together to achieve real, lasting change. Each organization has its own goals and niche but we come together around a common vision and collectively succeed.
  2. Teamwork is most successful when each member gets to apply their unique skill set. Yes, they’re all pretty fit and strong, but the Avengers are a hodgepodge of heroes, each with their own talents. They clash when there is ego and competition – who’s strongest, smartest, fastest? – but they succeed when they each respect one another’s unique talents and divide the work accordingly. The same is true for nonprofits, whether it’s within an organization or different organizations working together. Instead of one person or organization taking charge of all the work, it’s important to recognize the point above – you can’t do it alone – and divide the work such that each member gets to apply a unique set of skills. Be respectful of each other’s contributions and be humble enough to let others have ownership over something.
  3. Disagreements can be healthy discussion for growth. Because they’re big personalities, each with their own experiences and perspectives, the Avengers don’t always see eye to eye. Captain America and Iron Man/Tony Stark notoriously butt heads, eventually leading to a Civil War among heroes (to be featured in a film next year). In the latest Avengers film, Tony ends up disagreeing with not only Captain America, but Thor and Bruce Banner too, as he pursues his own agenda for creating a security system for the planet. But in the end, these disagreements are how Tony learns to set aside his personal goals and be a better teammate. Sometimes in the nonprofit world, we’re so consumed with our perspective that we don’t want to hear others’ opposing viewpoints. But being open to others’ views and ideas can help strengthen our own by showing us our weaknesses or by offering new questions, thoughts, and insights that help us develop and expand our own thinking. Smart organizations seek out new ideas, rather than dismissing them.
  4. The best intentions can have unintended consequences. In his attempt to protect the planet from another alien invasion, Tony works on the Ultron Project: an artificial intelligence system designed to act as a first line of defense. Of course, what he didn’t foresee was that the system would be so intelligent it would consider humans a threat to themselves, and therefore attempt to eliminate humankind. Similarly, nonprofits set out to solve problems but don’t always consider all the possible effects their actions may have. Taking a simplistic or myopic view of your work and its outcomes may leave you unprepared when things change or the unexpected happens. Organizations should invest in scenario planning – considering different possible futures you may encounter – and in risk assessments that lay out mitigation plans. The best way to ensure you are successful is to plan ahead and be prepared to adapt.
  5. Tackling problems means going after root causes. The Avengers track down Ultron with the aim of destroying him before he destroys humanity. As they attempt to stop their enemy, they also make sure to take care of all the people in danger’s way. But while the Avengers save innocent bystanders, they ultimately they go straight after the root of the problem. Nonprofits don’t always do the same, sometimes dealing with the immediate without ever tackling the root cause. While it’s important to handle the effects of the problem until the problem is solved, it’s far more important to go after the source of the problem, staving off any further negative effects. Effective organizations go after root causes to ensure a lasting solution to the problem they’re facing.
  6. Solving problems means following through to the end. Because Ultron was an artificial intelligence, it wasn’t enough to just destroy his physical form. In order to ensure that Ultron was eliminated for good, the Avengers had to destroy every last robot he built and controlled, so that there was no remaining piece of the program that could survive and rebuild itself. Nonprofits should work towards doing the same – going after the problem with the aim of eliminating it. Not reducing it and not just treating the symptoms of it but pursuing the eradication of the source of the problem. It isn’t going to be easy, but hey, neither is anything worth doing. Organizations should be making progress towards reducing the problem, not just keeping it at bay.

So go ahead. Mock me for liking comic books and superheroes. But don’t dismiss these important lessons for anyone aspiring to tackle societal problems and creating lasting change….

fundraising, logic model, messaging, strategy

Are you including funds for M&E?

With limited time and resources, nonprofit organizations are often forced to make tough decisions. Given the importance of the work they do – to those who do the work, to the constituents they serve, and to their donors and funders – it takes priority over everything else.

In fact, nonprofits are so busy trying to get the work done that when it comes to monitoring and evaluation (M&E), they often choose to do the work rather than stop to reflect and measure their impact. The result is a focus on outputs (the products or services provided, e.g. number of trees planted) rather than outcomes (a change in the way people or the system behave, e.g. reduced rates of deforestation). After all, it’s easier to measure what the organization has done in terms of activities than what it has achieved in terms of impact.

The problem is that M&E is an essential part of doing the work well. At the most fundamental level, measuring the work helps an organization to learn and improve the work it does, so it can do it both more efficiently and more effectively. It also helps an organization make the case that it does in fact make a difference in the world – that its work is actually solving a problem and it isn’t just spinning its wheels with no lasting effect. Organizations who take the time to monitor and evaluate their work also demonstrate a sophistication in their management and approach that helps them to rise above organizations who don’t prioritize learning and growth. And all of this strengthens an organization’s case for support, which in turn helps it attract more funding for its work.

Of course, even if an organization wants to monitor and evaluate its work, it doesn’t always have the resources to invest in it. Donors provide funds to get the work done but not to do the M&E required to improve it. While an organization can do some basic M&E on a tight budget, it certainly helps to have some financial support – as well as the encouragement from donors.

And donors should be investing in an organization’s M&E. If nothing else, monitoring and evaluating the work helps donors to see and understand the result of their investment. For foundations who have broader strategies to solve a problem, it can help them to measure their progress towards achieving their own goals in the field, as well as better understand what’s happening in the field in ways that could inform their own strategies. And instead of running their own evaluations, foundations should have the grantees do the M&E, for two reasons: (1) because they’re out in the field and closer to the work, grantees can provide more accurate information; and (2) it builds grantees’ own capacity to do M&E and to continuously learn and improve their work.

Since M&E is so valuable to both organizations and their funders, it should be a part of project budgets. M&E can be a line item for a dedicated staff member, or a consultant to help design a system the organization itself can run, or even just a percentage of the grant. Organizations should be asking for M&E funds, and funders should encourage their grantees to include M&E in their project budgets.

The best part about building M&E into the project budget is that it sets expectations that M&E is a part of doing the work – not something apart from it. Because it is part of the work: learning and evaluation is an important feedback loop that keeps the work on track, focused, and ultimately effective.

There shouldn’t be a choice between doing the work or measuring it. They should be one and the same.