SOCAP10: The Challenge of Moving Beyond Metrics – Part II

The Challenge of Moving Beyond Metrics – Part I

 

In my first post on moving beyond metrics, I reflected on the challenge of moving beyond the fixation of assessing nonprofits based on their overhead ratios and executive compensation. New assessment frameworks, championed by organizations such as Charity Navigator, GuideStar, GiveWell, GreatNonprofits, and Philanthropedia, will promote and embrace new ways to evaluate nonprofits and identify nonprofits. 

These new assessment frameworks will be discussed and used at a SOCAP10 session led by Ken Berger of Charity Navigator, Andrew Wolk of Root Cause, and Elie Hassenfeld of GiveWell. That session will review three analytical approaches to assessing DC Central Kitchen (per this post by Sean Stannard-Stockton). In Ken’s case, for example, the session will allow him to clarify the new approach to rate a nonprofit on its financial strength, accountability, and effectiveness.

The first challenge I noted deals with educating the nonprofit community, its leaders, and donors about the shift in focus away from financial metrics. To my disappointment, I keep hearing about organizations struggling to address overhead concerns. My second challenge deals with assessing nonprofit performance in the absence of full and real-time information.

I’ve been in the business of reviewing nonprofits as potential grant recipients for most of my career. As a grantmaker and steward of public and private funds, I’ve had to make sound recommendations and decisions on how to best allocate scare resources while maximizing impact. To that end, I’ve had access to a lot of data related to nonprofit performance and financial positions – often data not readily available in the public domain. And yet, I’ve never felt fully comfortable that the data at my disposal provided an accurate and current assessment of the organizations and that the investments I was going to recommend would maximize community impact

Most financial indicators fail to provide a real-time picture of organization. For example, most financial indicators available online reflect data that are 18 – 24 months old. As such, the data presented online – whether it is an organization’s Form 990 or an assessment of that data – reflects inputs that are somewhat stale. For example, much of the current data online reflect 2008 information. The economic recession’s impact – the loss of donors, the drop in fundraising, the massive staff layoffs and transitions – is not yet fully reflected in the data.

Similarly, performance outcomes are also a look back and don’t provide a full real-time overview of the organization. For example, let’s take an organization that has had a tremendous success in delivering services and has generated a track record of success. One could presume that an investment in that organization will generate additional high community impact. But that organization’s track history and performance outcomes won’t reveal that it is a dysfunctional internal mess that managed to achieve strong performance metrics despite its limitations. Nor will it tell you if those achievements were attained because of a visionary and talented person at the organization who has since left that nonprofit for a new enterprise.

In both examples, I could make a funding recommendation that would end up failing to generate the expected community impact.

How can we reasonably expect that an individual donor looking to make a charitable gift to an organization would be in a position to identify the best performing nonprofit organizations? Whether in looking at only financial metrics or examining a nonprofit’s effectiveness in achieving outcomes or a combination of both, the prospective individual donor still faces three major issues:

  1. The data tend to be old;
  2. The data don’t provide a complete informative picture of the organization; and
  3. They lack simple, digestible information to inform their decisions (see “Money for Good” report (PDF) by Hope Consulting). We still need to translate the performance outcomes into information that can be used easily by individual donors – regardless of their aptitude or interest to research nonprofits.

Without real-time context and information that effectively communicates the organization’s data, financial indicators and performance outcomes will continue to serve as imperfect proxies for identifying high impact nonprofits. To quote an excellent post by Jeff Stanger, “data are not information.” They need context and information to be truly useful.

So, while I applaud the move toward performance outcomes as assessment tool to identify high performing and impactful nonprofits, that move does not represent the ultimate panacea for identifying worthwhile organizations to support. Many individual donors and grantmaking institutions want to reduce the risk exposure of their investments by funding organizations that would most likely have strong performance in delivering expected outcomes. And yet, to paraphrase a famous stock market adage: past performance by a nonprofit is no guarantee of future result.

Perhaps what we ultimately need is a well-informed and real-time crowdsourcing operation to support donor looking to provide funds to high impact nonprofits. Moving from financial metrics to performance outcomes might be a step in that direction, but only a step. Hopefully, the SOCAP10 panel on Beyond Metrics would be able to comment on this challenge as well.

Disclaimer: This post is part of a series of posts on the SOCAP10 conference. As part of the blogging coverage for the conference, I was able to register to attend at a discounted rate. A version of this post appears on the SOCAP10 blog.

Tags: , , ,