Over the past year, I have pushed dozens of organisations and businesses in their efforts in impact evaluation. We have gone through realizations, several inspiring and complicated moments at various tables. Two questions seem to rise above others: how to proceed with evaluation, and how do you actually measure the impacts?
I have earlier, e.g. in my Conquering a mountain -column, offered tips on where to start the evaluation and how to utilize it in communications and marketing, too. I will continue on the same topic of the phases and measuring in evaluation.
Preparation, where do you want to climb?
Prior to conquering a mountain you must set clear targets.
- Why and to whom is evaluation done? What type of information is called for to convince these people, whose perspective and benefits shall we choose to evaluate?
- What is the impact goal of the product, service or concept to be evaluated, in other words, whose life are we trying to change? If we succeed in it, how are the changes replicated in society: what headlines does your solution provide?
Choose the goal according to these targets, a route that fits your condition and a good team to accompany you in climbing the mountain. It is a different matter altogether to utilize impact evaluation in developing operations or in piloting a new concept, than it is to seek new material for marketing communications or provide evidence for investors.
Base camp, what does the route forward look like?
When you have a clear idea of the goals for impact evaluation as well as for the solution being evaluated, you start drafting an impact model for the solution. What is the root cause of the problem you are tackling? How are the desired changes made, and why? Which changes occur instantly and how are they replicated in other contexts?
The impact model shows how the solution works and who will benefit from it.
The most important thing is that the cause and effect relations that the model describes are based on researched information, as to make sure we are not dealing with an impact guess. There are many ways to outline and visualize the impact model. The impact chain is probably the most common one, but an impact tree may also be useful on its side or as an alternative.
Towards the peak, with appropriate tools available.
Evaluation simply means collecting and analysing information, – i.e. research. The aim is to prove the impact model either right or wrong, and to help refine the evaluated solution further. You can reach the peak with a good set of indicators and plan for data collection.
A good set of indicators…
- follows you on your way to the selected impact goal.
- covers all stages of the impact model from input to impacts.
- yields information on changes detectable in short- and long-term.
- yields information from the perspective of various stakeholders.
- contains subjective and objective indicators.
The set of indicators should suit the goals of impact evaluation and the impact model alike. Who was it again that we collect data for, and how can the changes that we aim at through the operations be discerned and verified? If some data is not needed, you shouldn’t collect it!
You can find some signposts to sources of reference data and to building up an indicator set in this text. A simple research plan that is suitable for impact evaluation can be compiled like this:
Impact evaluation should not be one single venture to Mount Everest – being able to tick another point on your bucket list. At best, evaluation is a circle that sometimes expands, sometimes contracts, and gives you resources to improve operations, taking you to higher peaks still.
Enjoy your trip!