The word ‘measuring’ takes our thoughts to engineers, economists, calculations and various gadgets. But what is the level for welfare, pH meter for human relationships or supporting rail detector for the faith in future?
Crap won’t get better by measuring it
Excuse my French, but this spot-on expression was actually made in a training I attended. Measuring and impact evaluation should interest everyone who claims to provide something positive to society. Measuring is most of all a way to ensure that the output and results of one’s activities are not crap.
Are your claims realized, and why? Under what conditions are the set goals met? Are you simultaneously sticking out your behind to the other direction and causing problems somewhere else?
What to measure?
- Base the indicators on the impact model – make sure you have one, e.g. in an impact tree?
- Don’t do anything until you have set a crystal clear goal for impact verification: what you are doing, why and to whom. Do not collect information that no one will use.
- Societal change almost always starts from the change in an individual’s behaviour. For the change to be lasting, the activity must affect the person’s motivation and abilities and give him a chance to act differently. These elements should also be reflected in the measuring tape.
- Start with the most important thing with regard to operations and evaluation.
- Do not forget the process indicators. They help to monitor the direct changes that occur and whether the daily activities bring you towards the impact goals.
- Verifying change can be done e.g. with reference frames, follow-up or before and after measurement. Sometimes (i.e. often) the comparison point must be found in the applicable demographic data and studies
- Do not reinvent the wheel. Use the studies and data sources in your field as the model for research frames, question sets and questions and their answer options.
- Combine various data collection models and indicators: Collect subjective, objective, qualitative, quantitative data, and don’t forget to choose one or two indicators requiring a longitudinal approach .
- Start by experimenting and in a sufficiently restricted scale. Make – and fix – your mistakes quickly.
Chicken or egg?
Impact evaluation and measurement is still a forerunner’s job. Forerunners must muddle along in partly unexplored terrain and clear the way for others.
When demand, i.e. investors and buyers, starts coming in, the one with the goods in stock wins.