When
the American philanthropist John D Rockefeller famously said,
“Next to doing the right thing, the most important thing
is to let people know you are doing the right thing”
you can imagine that a cynical eyebrow or two was raised,
as this idea doesn’t really mesh with traditional views
on charitable endeavour. But the man had a point.
In the modern third sector, there isn’t much room for
quietly getting on with your work and assuming that that work
will speak for itself. Aside from the actual regulatory reporting
requirements with the Charity Commission, donors and funders
are increasingly looking for evidence of the impacts achieved
by the charities to which they are giving money. And charities
themselves are obviously interested in effectively assessing
their own outcomes and impacts, allowing them to see where
they are performing well, and what areas need work.
One way of doing this is through a relatively new (in the
UK) reporting method known as social return on investment.
Similar to ROI calculations carried out by businesses to justify
their activities to shareholders, SROI provides a comprehensive
analysis of the effectiveness of a charity or programme of
activities, and ultimately puts a monetary figure on the amount
of ‘good’ provided to society.
Originally an American concept, it was born in the 1970s when
the US government mandated ‘benefit cost analysis’
for federal contracts. This was much furthered in the mid-nineties
by the San Francisco-based REDF (which invests in early stage
social enterprises) as a means of determining the social impact
of the projects it was funding.
And it has been gaining ground in the UK in the last few years,
driven by organisations such as think tank the New Economics
Foundation (Nef), and more recently the Vodafone UK Foundation
in conjunction with consultancy the Corporate Citizenship
Company.
Top
SROI, the basics
At its core, SROI is a robust reporting tool which attempts
to put a value on the work an organisation is doing, relative
to a given amount of investment. The actual process involves
analysis of inputs, outputs, outcomes and impacts leading
to the calculation of a monetary value for those impacts,
and finally to an SROI rating – a monetary value relative
to investment in the project (see below for information
on the full procedure).
The ultimate result is a hard figure of, say, 3:1; so for
every pound invested in a project, society is three pounds
better off. This could, put simply, represent a person gaining
employment through a charity’s efforts, thereby increasing
tax revenue and decreasing state support, allowing that
figure of societal benefit to be generated.
Arguably, though, this end figure is less important to an
organisation undertaking an SROI analysis than the actual
process used to achieve it. According to Lisa Sanfilippo,
head of measurement and evaluation at Nef, that process
itself is an effective technique for charities to better
understand and report their outcomes, and the framework
allows charities to focus on where value is actually being
created for stakeholders.
She points out that in some reporting frameworks there is
a potentially more “diffused” idea of value.
However, she says: “The principal in SROI is that
of materiality – what’s going to help stakeholders
to make a better informed decision about how effective that
charity is in delivering targeted benefits to the people
that matter most.”
Andrew Wilson, director of the Corporate Citizenship Company,
sees SROI as having three broad aims: showing how a charity
can improve the management and effectiveness of what it
is doing; as a reporting tool for external funders who are
looking for evidence of the impact of their investment;
and also as a means for charities to learn from each other
by sharing data derived from the exercise.
The concept was discussed in detail earlier this year at
the Institute of Fundraising annual convention, where Wilson
and the Vodafone UK Foundation’s director Sarah Shillito
ran through a new project they were undertaking in response
to what appears to be a shortfall in current SROI research.
The majority of existing research, says Wilson, mainly examines
employment and improving employability, which is much easier
to measure than areas such as prevention. “What we’re
saying is, how can you put a financial value on the positive
impact of somebody avoiding homelessness [for example].
It can be done, it’s just slightly more difficult
to get the data, but it is the same principles.”
The end goal of this is a straightforward SROI toolkit that
anyone can make use of, which Shillito says aims to be highly
user friendly as “there is an awful lot of great work
and research out there that sits on the shelf and gathers
dust”.
Not so straightforward
During the Q&A session at the IoF presentation, however,
the audience seemed somewhat bemused. The issue of comparing
non like-for-like charities was a major area of contention,
as well as putting hard figures on ‘softer’
missions like improving self-esteem (which the Vodafone
project is addressing). There was also fear that those hard
figures could be misused by funders making uninformed decisions,
potentially based on an SROI league table of some description.
Andrew Dunnett, director of Vodafone Group Foundation has
his own questions about comparing non like-for-like organisations.
He says that every funder is keen to improve measurements,
and any tool that assists in that process is welcomed. “But
how do you compare two NGOs, one of a global reach and one
from a start-up, because we fund both of those,” he
asks. “How would you develop some sort of standard
list or indices that can really reach both of those and
give an accurate portrayal?”
Addressing this question, Nef’s Sanfilippo says that
at present you don’t compare an organisation trying
to bring people back into work, with one that increases
self-confidence, for example. “You compare apples
to apples and oranges to oranges.”
She also says that concern over the difficulty of softer
targets is one of the most common misconceptions in measuring
outcomes among charities. People believe that because something
is not as tangible it is harder to measure, she says, when
in fact it just requires a different method of measuring
it. “At the moment, the difficulty around SROI is
that charities feel apprehensive about measuring outcomes
because they don’t quite know how to do it.”
Potential misuse
But what of the potential for funders to misuse the hard
figures derived from SROI reporting? This fear was evident
at the IoF convention, particularly in terms of oft maligned
commissioners trying to ‘maximise efficiency’
in contract decisions. But is this fear justified?
“There is a potential danger that that would happen,
but I think most people who are seriously considering supporting
projects want to look beyond just a single metric,”
says Wilson. “So it’s an understandable fear
but I think it’s unfounded.”
Les Hems, director of development and external affairs at
GuideStar agrees, saying that any raw figures need to be
accompanied by the story behind how those figures were derived
and this, hopefully, is what funders will be looking at.
“As long as the principle of numbers and narratives
is sustained, then any funder, whether it be statutory or
grant maker, should be able to understand those numbers
and compare them in that context.”
Shillito, as well, says that she isn’t really looking
at the benefit of SROI as a single figure at the end of
the process. “I’m looking at this as much the
same as when they say ‘one-in-four marriages ends
in divorce’ or ‘if you smoke you are ten times
as likely to have lung cancer’ – I see it as
much more like that, where you can pick out stats rather
than having one headline figure for the whole of your work.”
Top
What the funders say
Other funders are quick to point out that fears over future
misuse of SROI figures are unjustified, and that funding
decisions would never be as simple as a straight comparison
of SROI ratings. They agree with Hems’ description
of numbers and narratives, as the decision making process
is just too complex.
The Association of Charitable Foundations, for example,
says that discretionary grant-making is about backing risks
and people, using a range of practices based on “informed
judgement” rather than a single analysis technique.
As such, it believes SROI might be a useful tool but is
unlikely to be seen by grant-makers as the sole means of
determining the effectiveness of an organisation or the
value of a particular funding application.
As its chief executive David Emerson points out: “In
the sense that along with all of society our members are
naturally interested in knowing whether their funding has
been effective, we could see that this would become one
of many possible tools they might sometimes find useful.”
Vodafone Group Foundation’s Dunnett says that his
organisation looks at things on an individual basis. Once
they know a project is in line with their funding philosophy,
it is then a matter of examining track record, capacity
to deliver, individual KPI, and the reporting system in
place. “It’s obviously of interest if there
is some type of Kitemark or position or recognition of a
charity’s capacity in their recent past to deliver,”
he says, “but I think the primary focus of our grant
making approach would be the values and the individual organisation
itself.”
This position is furthered by Richard Gutch, chief executive
of Futurebuilders. He says that a favourable SROI rating
is one of many ways of assessing the effectiveness of an
organisation, but there is also good governance, comprehensive
business plans, and the quality of HR and IT systems to
name a few.
But is this view shared by commissioners? Wouldn’t
a single, comparable figure derived from an SROI analysis
be the Holy Grail of government funding decisions and services
commissioning? Well, according to commissioners, no.
Doug Forbes, from the newly established Institute of Commissioning
Professionals, says: “I have a feeling that the voluntary
sector doesn’t really understand the rules by which
the public sector has to play.”
He points out that European law demands transparency, mutual
recognition and equal treatment, and these must apply to
any group being offered services via contract. “Therefore
SROI reporting can only be one of a number of measures for
evaluation,” he says. “Using this for one sector
may be seen to unfairly exclude others. If it provides a
fair, objective, relevant and transparent measure, it can
be used. However, there will be an overhead which has to
be funded. Where are the pilot schemes and how will its
advantages be assured?”
This point is furthered by Andy Fox, a commissioner for
East Renfrewshire in Scotland, who says: “It seems
a fairly complex process and I wonder whether the ‘science’
behind it is robust enough to provide real value or if it’s
open to a too subjective analysis.”
He says that he would be reluctant to put new analytical
processes into the Scottish commissioning context until
he is confident that a national commissioning framework
and guidance exists into which an innovative measurement
system, including a reporting tool like SROI, could be built.
Standardisation
Fox’s concerns actually sum up the debate around SROI
quite well. As a tool for a charity to determine its own
effectiveness it is definitely powerful, but for external
funders to make comparative decisions based on SROI ratings
it is very early days, and may in fact never be a completely
viable option.
Futurebuilders’ Gutch says he has found that the accuracy
of ratings is affected by inconsistencies in the underlying
assumptions on which the model is based. Uncertainty about
where to set boundaries for the calculations results in
very wide confidence bands for the value of the end figures,
he says, and ultimately weakens the accuracy of the resulting
ratios.
Malcolm Hayday, chief executive of Charity Bank, furthers
this point, saying: “I’m not totally convinced
that we’ve all got it right yet.” He says that
at the moment there seems to be a lot of different systems
and ideas around SROI, and there isn’t necessarily
a common factor, which makes it quite difficult to compare
performance between organisations.
But is there a way to bring everyone in line? “I’m
not too sure you can all come into line because I think
different organisations, or different clusters of organisations,
have different objects and motivations, and therefore it
might be something that can only be reviewed among those
clusters,” Hayday says.
Lindsay Boswell, chief executive of the Institute of Fundraising
agrees with this, saying that with such a diverse sector,
obtaining a workable level of standardisation could only
be done among small subsets in any “intellectually
robust” way.
One possible solution could be some sort of official body
to ensure a level of standardisation across reporting. An
audience member at the IoF convention recommended the Fundraising
Standards Board undertake this role, but that doesn’t
seem likely in the near future.
The Corporate Citizenship Company’s Wilson agrees
that something will be necessary however. “I think
as and when this technique becomes more widely understood
and more widely used, there will be a need for some kind
of verification or accreditation of the process. A standards
body may be one way of doing it, or it may be a peer review
process.”
Going forward
So what for the future of SROI then? It seems that the fears
voiced at the IoF convention don’t really have much
validity, and that SROI analysis is probably going to become
much more mainstream.
Hayday says he sees it playing a greater role in Charity
Bank’s funding decisions going forward, and Futurebuilders’
Gutch believes that it has the potential to become a useful
tool in determining the effectiveness of organisations seeking
to deliver public services.
However, says Gutch, for this to happen, further work needs
to be done to find a much simpler way of using the SROI
model as, at present, SROI methodologies are far too complicated
to be operational at a practical level.
Hopefully the Vodafone UK Foundation’s toolkit will
go some way towards addressing this, but we’ll have
to wait until the end of the year to see what they come
up with. If, however, SROI better allows people to know
that you are ‘doing the right thing’ without
it being misused by those making funding decisions, then
surely an increase in its popularity will be the right thing
for the sector as a whole.
An SROI guide
If you’re itching to carry out an SROI evaluation
for yourself and can’t wait for the Vodafone toolkit,
Nef has step-by-step guidance available on its website for
download.
A very robust piece of work, it’s about 80 pages long
– and may be pretty daunting for the uninitiated.
It is definitely worth a look, though, if only to see what
the full process entails.
Visit www.neweconomics.org/gen/z_sys_publications.aspx
and click on Measuring real value: A DIY guide to Social
Return on Investment.
Top
To return to the September 07 features list click
here
|
