Learning from developing thematic tools

Erica Bertolotto, Consultant at NPC, reflects on her work developing three new tools to help organisations working on education, mental health or housing create a measurement framework with relevant outcomes, indicators and tools.

The Pathway strand of the Impact Management Programme (IMP) is all about helping charities and social enterprises get started with impact management and navigate the variety of advice available online. Within this, we know that it can be difficult to find thematic tools and guidance such as education, mental health or housing. A one-stop-shop thematic tool for different areas seemed like a good place to start—including theoretical guidance, practical activities to develop an outcomes framework, and signposting to relevant resources on the IMP website or elsewhere.

A framework for mental health and wellbeing organisations

The first such tool I developed is for mental health organisations. Guidance and activities are broken down into bite-sized chunks: starting with an introduction to user, engagement and feedback data, it then asks users to select the outcomes they’re working towards from a list and to prioritise a small number of outcomes from the ones selected, that the organisation will measure. Following this, key indicators and measurement tools are suggested for the priority outcomes, as well as guidance on when it’s appropriate to use validated tools and how to write a bespoke survey or interview guide. By the end of the document, an organisation will have identified their outcomes, indicators and potential tools.

To make sure the outcomes, indicators and tools recommended were right for mental health organisations, and that the guidance was practical to implement, I consulted with some leaders in the field.

  • Mind reviewed the tool in a lot of detail and provided invaluable feedback on aspects such as the language used by mental health charities, the suitability of outcomes and indicators, and the most commonly used validated tools.
  • Mosaic Clubhouse, Rethink Mental Illness and Maytree also reviewed the guidance and were very positive about the tool.

Everyone we consulted felt that standard outcomes, indicators and tools to pick from would be very useful and have the potential to save organisations lots of time and money. However they pointed out that sometimes outcomes are set by funders, or agreed together with funders, and it may therefore be harder for charities to decide outcomes independently.

Applying what we learned to frameworks for education and housing organisations

Given the feedback from these organisations and the appetite for practical tools like this, I created similar ones for the education and housing sectors, to enable organisations working in these areas to develop their outcomes frameworks. We’ve collected feedback on the proposed outcomes, indicators and tools from experts in these fields.

  • IntoUniversity added their notes on the usefulness and ease of obtaining each of the measurement tools in the education outcomes framework
  • Young Foundation gave feedback on the suggested outcomes and tools
  • Re-Entry shared their thoughts on when and how they would use the tool
  • Marcus Hulme from Big Society Capital advised on the housing framework based on BSC’s work in this area

These tools are a first step to enabling shared measurement: organisations working in the same sector measuring the same outcomes using the same tools. To make most of the tools, all the organisations using them would need to be able to store their data in a central place and benchmark their progress against others. Inconsistent demand, along with the cost of developing and hosting a digital platform for this are key barriers to implementing shared measurement. NPC has reflected on this in the past through their decision to end their Well-being Measure work.

Understanding demand for and use of these tools

Once the new IMP website is launched in early March the tools will also be available via the data diagnostic, a questionnaire that gives organisations tailored advice about what data they could collect and how to do that. It will be interesting to see how many organisations use the thematic tools and what they think of them…and whether there is demand for further developments in these areas, in the form or shared measurement or something else.

If you explore one of these tools and have feedback for us - please !

These tools have drawn on existing frameworks including Inspiring Impact resources, NPC outcomes map: mental health and the BSC outcomes matrix – mental health and wellbeing for individuals.