Our website uses cookies

Cookies help us to understand how you use our website so that we can provide you with the best experience when you are on our site. To find out more, read our privacy policy and cookie policy.

Manage cookies

Please review and manage the Cookie settings below. You can change these settings any time by clicking the "Cookie settings" link in the footer of the page

  1. Essential cookies:
    Necessary for enabling core functionality. The website cannot function properly without these cookies.

IE10 and below are not supported.

Contact us for any help on browser support

Opening Pandora’s Evaluation Box

by Dan McKillop, over 2 years ago
CLOSED: This discussion has concluded.

By Allyson Hewitt, JW McConnell Family Foundation Senior Fellow, Social Innovation at the MaRS Discovery District

April 26, 2016

“Revolutions in science have often been preceded by revolutions in measurement.” Based on the premise from Sinan Aral of the MIT Sloan School of Management

Jason Saul presented to a full-house as part of the MaRS Global Leaders series in April 2016 on his latest venture – the Impact Genome Project (IGP). A public-private partnership to code and quantify the “genes” of what works in social science. The audio of the presentation can be found below.

If you spend 5 minutes in the social impact sector you are sure to be asked, how do you know you are making a difference?

Skoll Forum 2014


Jason and his colleagues at Mission Measurement have been tackling this question by taking us from the current state: we are spending $400 billion to achieve social outcomes without any standard way to accurately measure ROI; the evaluation industry is in disarray; evidence is unstructured and unintelligible; and yet evidence is growing exponentially – it is just not readily accessible. We have no common language; no benchmarks that allows us to compare social programs; and ultimately no predictive data meaning we can’t forecast before we invest. This is what Mission Measurement calls the black box problem.

Yet other sectors have predictive data and use it to increase their impact: think credit scores, the human genome or even Netflix. The music industry has cracked this code with Pandora and their Music Genome Project, the original inspiration for the Impact Genome Project. Jason approached Nolan Gasser, the architect of the Music Genome Project, and together they embarked on a journey of discovery asking one question: Can we not do for social programs what Pandora did for music?

As it stands the Impact Genome Project comprises 11 total genomes: education; economic development; public health; youth development; international development; human services; criminal justice; sustainability & environment; science & technology; arts; and culture & identity. With 132 common outcomes. The goal of IGP is to produce new benchmarks such as efficacy rate; expected outcomes; and cost per outcome. It is an open data project with advanced analytics available via subscription.

The IGP intends to create a more level playing field by:
  • Democratizing evaluation
  • Replacing guessing with data
  • Learning systematically across the sector
  • Unleashing innovation and creating twice the impact with half the cost


It is an audacious goal and yet the future is here. The UK government is already moving to pay for outcomes and have created a What Works Centres, a network of centres to “support more effective and efficient services across the public sector at national and local levels.” Our own governments are not far behind with the Centre of Excellence for Evidence-based Decision Making Support at the Government of Ontario, which was part of Minister Deb Matthews’ mandate letter.

Why Canada? We have supportive genetic infrastructure:
  • Un-entrenched philanthropic institutions
  • Integrated and collaborative philanthropic sector
  • Government prioritizing evidence and value for money
  • Institutions willing to lead
  • Access to top talent/academic institutions
  • Systems-thinking expertise


We are interested in what you think. Does this seem like a way to get ahead of the inevitable move to pay for outcomes? Can we work with funders to make this approach the standard, not the only way forward but one that is “directionally correct”? What are your concerns, if any?

Please let us know and help us determine how we can get to a better place around demonstrating our impact in a world that needs us to use all our talents to tackling our complex challenges.



Share on Facebook Share on Twitter Share on Linkedin Email this link

Consultation has concluded