“For every complex problem, there is an answer that is clear, simple, and wrong.” – H.L Mencken
Evaluating Social Labs has to be done in the context of the complexity they seek to address.
There’s an emerging field of Evaluation which is grappling with how you design and develop appropriate evaluation programmes when you’re working in the field of complexity.
Being too reductionist can drastically underestimate the kind of value and change that a Social Lab is creating. The multiple ‘Capitals’ generated by a Social Lab help to name some of the areas that might be paid attention to:
We also wrote a little about hacking your way to world class Evaluation last year, which is worth a read to get some principles in place to set up well.
Take a deep dive into Evaluation for Complexity
If you want to dive deeper into the realm of developing an evaluation program for Social Labs, or other interventions which are operating on a complex problem, then we also suggest checking out FSG’s Report entitled “Evaluating Complexity : Propositions for Improving Practice”. Here’s a slide which sums up the 9 tenets of Complexity and how to address it with Evaluation:
Read “Evaluating Complexity : Propositions For Improving Practice” here.
See An Example
We’ve been writing about the importance of Storytelling in the Social Labs space, because we feel it’s a vital way of communicating the deeply human experience of a Social Lab.
That said, we also have been scanning the horizon for great examples of communicating the practice and value of intervening in complex challenges. One of our favourite reports is from Thicket Labs in New York:
In Thicket’s Case Study on Chinatown Youth Initiatives, they lay out some really interesting graphics which explore how the CYI program compares to other benchmarks of programs/initiatives which affect their lives. In this graphic, you can see visuals which show CYI Vs School, based on data collected with cognitive mapping techniques.
We feel this is a relatively novel way to communicate the impact of initiatives working in a complex, dynamic environment. However, with most forms of impact evaluation, this is just a snapshot in time.
The Future of Evaluation for Complexity?
Where the future heads, we don’t know – but we do recognise the importance of evaluation programs which actively thrive in complexity (FSG gave some good starting points on how to achieve this).
As for communicating the outcomes of these Evaluation programs, it seems like there’s a long way to go to moving beyond snapshots in time, and instead begin to present real time reporting on how initiatives are creating impact.
One example of how the multi-capital approach is being used in Evaluation and Reporting is in Integrated Reporting, which is well worth a look.
In theory, with technological advances, we should be closer to seeing some form of Impact Intelligence Dashboard emerging to help us become more proactive and responsive to the dynamics of complex challenges.