Posts filed under ‘Campaign evaluation’
Visualizing Information for Advocacy
Just came across this interesting guide(pdf) to using visuals for advocacy from the Tactical Technical Collective – here is an explanation from the authors:
Visualising Information for Advocacy(pdf): An Introduction to Information Design is a manual aimed at helping NGOs and advocates strengthen their campaigns and projects through communicating vital information with greater impact. This project aims to raise awareness, introduce concepts, and promote good practice in information design – a powerful tool for advocacy, outreach, research, organisation and education. Through examples, the booklet demonstrates how to use innovative visual graphics to tell a complex and powerful story in a snapshot.
Evaluation of communication activities of international and non-governmental organisations: A 15 year systematic review
As part of my PhD studies, I have undertaken a systematic review of how international and non-governmental organisations are evaluating their communication activities. I’m presenting a summary of this today at the European Evaluation Society Conference in Helsinki, Finland. Below are the slides, hope you find them interesting.
New advocacy evaluation guide
Bond, the UK alliance of NGOs, has produced an interesting guide on advocacy evaluation:
Assessing effectiveness in influencing power holders (pdf)
The guide looks at the challenges of influencing power holders (usually done through activities grouped under the umbrella of “advocacy”) but comes to the conclusion that evaluation is feasible:
it is possible to tell a convincing story of an organisation’s contribution to change through their influencing and campaigning work by breaking down the steps of the process that led to change, and looking at how an organisation has created change at each step.
The guide also sets out these steps and provides examples of advocacy evaluation tools from NGOs including Oxfam, CARE, Transparency International amongst others.
Understanding public attitudes to aid and development
Here is a fascinating research paper on Understanding public attitudes to aid and development (pdf) from the UK-based ODI and IPPR.
Relevant to monitoring and evaluation, it recommends:
“Campaigns should do more to communicate how change can and does happen in developing countries, including the role aid can play in catalysing or facilitating this change. Process and progress stories about how development actually happens may be more effective communication tools than campaigns focused straightforwardly on either inputs (such as pounds spent) or outputs (such as children educated).”
This is a weakness of campaigning about development and aid, in that the steps towards change are not explained – the so-called “theory of change” – “if we do that – it will lead to that” – is a mystery – for the public and often those running the programmes have not always thought it through either…
Thanks to the Thoughtful Campaigner blog for bringing this to my attention.
advocacy evaluation: influencing climate change policy
Often I don’t get to share the findings of the evaluations I undertake, but in this case of an advocacy evaluation, an area that I’ve written about before, the findings are public and can be shared.
I was part of a team that evaluated phase 1 of an advocacy/research project – the Africa Climate Change Resilience Alliance (ACCRA). ACCRA aims to increase governments’ and development actors’ use of evidence in designing and implementing interventions that increase communities’ capacity to adapt to climate hazards, variability and change. Advocacy plays a large role in trying to influence governments and development actors in this project. You can read more in the Executive_Summary (pdf) of the evaluation findings.
The evaluation also produced 5 case studies highlighting successesful advocacy strategies:
- Capacity building and district planning
- Secondment to a government ministry
- Reaching out to government and civil society in Uganda
- Disaster risk profiling in Ethiopia
- Exchanging views and know-how between ACCRA countries
The case studies can be viewed on the ACCRA Eldis community blog (n.b. you have to join the Eldis community to view the case studies, it’s free of charge).
To disseminate the evaluation findings widely we also produced a multimedia clip, as featured below.
Advocacy evaluation – top resources
Today I spoke to the students of the Executive Certificate of Advocacy in International Affairs at the Graduate Institute of Geneva on advocacy evaluation. I promised the students to list the top resources I’d recommend on advocacy evaluation, here they are:
Practical Guide to Advocacy Evaluation from Innovation Network (pdf)>>
Guide on measuring advocacy and policy (pdf) from the Annie E. Casey Foundation
“A guide to monitoring and evaluating policy influence (pdf)” of the UK-based Overseas Development Institute describes the different approaches to evaluating policy influence.
“Advocacy Impact Evaluation” (pdf) by Michael Q. Patton – an interesting case study on influencing the US Supreme Court.
“Lessons in Evaluating Communication Campaigns: Five Case Studies” from the Harvard Family Research Project looks at evaluating advocacy campaigns ranging from gun safety to emmissions (ozone) reduction.
Measuring the impact of NGO programmes
An ongoing debate focuses on how NGOs can measure the impact of their work. The International Initiative for Impact Evaluation (3ie) and Oxfam have recently produced a very interesting paper on this subject:
Using examples from campaigns and other programmes, the paper sets out the challenges and options in evaluating impact and proposes four options for improving impact evaluation:
1) partnering with research institutions to rigorously evaluate “strategic” interventions;
2) pursuing more evidence informed programming;
3) using what evaluation resources they do have more effectively;
4) making modest investments in additional impact evaluation capacity.
View the paper (pdf)
Ten takeaways on evaluating advocacy and policy change
The Harvard Family Research Project produce some excellent material on advocacy evaluation.
From their newsletter(pdf), here are ten takeways on evaluating advocacy and policy change:
1.Advocacy evaluation has become a burgeoning field.
2. Advocacy evaluation is particularly challenging when approached with a traditional program evaluation mindset.
3. The goals of advocacy and policy change efforts—that is, whether a policy or appropriation was achieved—typically are easy to measure.
4. Many funders’ interest in advocacy evaluation is driven by a desire to help advocates continuously improve their work, rather than to prove that advocacy is a worthy investment.
5. Advocates must often become their own evaluators. Because of their organizational size and available resources, evaluation for many advocates requires internal monitoring and tracking of key measures rather than external evaluation.
6. External evaluators can play critical roles.
7. Context is important.
8. Theories of change and logic models that help drive advocacy evaluation should be grounded in theories about the policy process.
9. Measures must mean something.
10. Evaluation creativity is important.
Advocacy campaigns and policy influence
Influencing policy is often an aim of many advocacy campaigns – the notion of trying to bring about change in the policy of governments, the private sector or even international organisations (e.g. UN). Here are two interesting publications in this area:
“Pathways for change: 6 Theories about How Policy Change Happens (pdf)“ of the US-based Organization Research Services describes different theories as to how policy change can occur – interesting reading for those trying to influence policy.
“A guide to monitoring and evaluating policy influence (pdf)” of the UK-based Overseas Development Institute describes the different approaches to evaluating policy influence – i.e. how you can evaluate your efforts to influence policy.
Both publications are worth a read if you are interested in policy influence and advocacy campaigning.
Online campaigning handbook
Here is a newish (well I just discovered it..) online campaigning handbook (pdf) from Publiczone.
Point 10 of the handbook “Keeping track of what you are doing” focuses on monitoring and evaluating online campaigns. Here is an extract of what they recommend:
Effective monitoring and evaluation can make the difference between an average and an amazing campaign. Monitor and evaluate as you go along and you’ll keep finding new opportunities to optimise your campaigning…The trick is to design your evaluation before you start, paying close attention to how you are going to collect data. Too often, charities leave evaluation to the end, only to discover they can only form a patchy picture of their campaign due to an absence of data.
