Posts filed under ‘Advocacy evaluation’
Insights into NGO advocacy evaluation
A group of leading NGOs have published a comparative study (pdf) on what they are doing in practice on advocacy evaluation. Participating NGOs are: ActionAid International, Amnesty International, Bread for the World, CARE USA , Greenpeace International, ONE, Oxfam and the Sierra Club.
Following is a summary from the study:
For organizations committed to social change, advocacy often figures as a crucial strategic element. How to assess effectiveness in advocacy is, therefore, important. The usefulness of Monitoring, Evaluation and Learning (MEL) in advocacy are subject to much current debate. Advocacy staff, MEL professionals, senior managers, the funding community, and stakeholders of all kinds are searching for ways to improve practices – and thus their odds of success – in complex and contested advocacy environments.
This study considers what a selection of leading advocacy organizations are doing in practice. We set out to identify existing practice and emergent trends in advocacy-related MEL practice, to explore current challenges and innovations. The study presents perceptions of how MEL contributes to advocacy effectiveness, and reviews the resources and structures dedicated to MEL.
Visualizing Information for Advocacy
Just came across this interesting guide(pdf) to using visuals for advocacy from the Tactical Technical Collective – here is an explanation from the authors:
Visualising Information for Advocacy(pdf): An Introduction to Information Design is a manual aimed at helping NGOs and advocates strengthen their campaigns and projects through communicating vital information with greater impact. This project aims to raise awareness, introduce concepts, and promote good practice in information design – a powerful tool for advocacy, outreach, research, organisation and education. Through examples, the booklet demonstrates how to use innovative visual graphics to tell a complex and powerful story in a snapshot.
BetterEvaluation – great resources for M&E
Here is a new website (well, new for me), that I recently discovered:
“An international collaboration to improve evaluation practice and theory by sharing information about options (methods or tools) and approaches. “
There are many resources on the website, for example, if you are interested in advocacy evaluation, there are useful resources on “process tracing”, a useful method for this area.
New advocacy evaluation guide
Bond, the UK alliance of NGOs, has produced an interesting guide on advocacy evaluation:
Assessing effectiveness in influencing power holders (pdf)
The guide looks at the challenges of influencing power holders (usually done through activities grouped under the umbrella of “advocacy”) but comes to the conclusion that evaluation is feasible:
it is possible to tell a convincing story of an organisation’s contribution to change through their influencing and campaigning work by breaking down the steps of the process that led to change, and looking at how an organisation has created change at each step.
The guide also sets out these steps and provides examples of advocacy evaluation tools from NGOs including Oxfam, CARE, Transparency International amongst others.
advocacy evaluation: influencing climate change policy
Often I don’t get to share the findings of the evaluations I undertake, but in this case of an advocacy evaluation, an area that I’ve written about before, the findings are public and can be shared.
I was part of a team that evaluated phase 1 of an advocacy/research project – the Africa Climate Change Resilience Alliance (ACCRA). ACCRA aims to increase governments’ and development actors’ use of evidence in designing and implementing interventions that increase communities’ capacity to adapt to climate hazards, variability and change. Advocacy plays a large role in trying to influence governments and development actors in this project. You can read more in the Executive_Summary (pdf) of the evaluation findings.
The evaluation also produced 5 case studies highlighting successesful advocacy strategies:
- Capacity building and district planning
- Secondment to a government ministry
- Reaching out to government and civil society in Uganda
- Disaster risk profiling in Ethiopia
- Exchanging views and know-how between ACCRA countries
The case studies can be viewed on the ACCRA Eldis community blog (n.b. you have to join the Eldis community to view the case studies, it’s free of charge).
To disseminate the evaluation findings widely we also produced a multimedia clip, as featured below.
Advocacy evaluation – top resources
Today I spoke to the students of the Executive Certificate of Advocacy in International Affairs at the Graduate Institute of Geneva on advocacy evaluation. I promised the students to list the top resources I’d recommend on advocacy evaluation, here they are:
Practical Guide to Advocacy Evaluation from Innovation Network (pdf)>>
Guide on measuring advocacy and policy (pdf) from the Annie E. Casey Foundation
“A guide to monitoring and evaluating policy influence (pdf)” of the UK-based Overseas Development Institute describes the different approaches to evaluating policy influence.
“Advocacy Impact Evaluation” (pdf) by Michael Q. Patton – an interesting case study on influencing the US Supreme Court.
“Lessons in Evaluating Communication Campaigns: Five Case Studies” from the Harvard Family Research Project looks at evaluating advocacy campaigns ranging from gun safety to emmissions (ozone) reduction.
A pragmatic guide to monitoring and evaluating research communications using digital tools
The approach taken relates online measurement tools to four levels of assessing influence of communications on policy (an aim of research communications):
- Management, outputs, uptake, outcomes and impact.
The last level, outcomes and impact is of course the hardest to measure with digital tools. But I think if you have access to your target audiences, this can be done through in-depth interviews or more simply through email surveys to ask how they have used the research products – which can give then provide an indication of the role they have taken in influencing policy.
EU Manual: Evaluating Legislation and Non-Spending Interventions in the Area of Information Society and Media
A very interesting manual published by the European Union:
Despite the wordy title, the manual is really about how to evaluate the effects of legislation and initiatives taken by governments (in this case the regional body – EU).
The toolbox at page 72 is well worth a look.
The Elusive Craft of Evaluating Advocacy
Here is an interesting article “The Elusive Craft of Evaluating Advocacy” (pdf) that examines some of the challenges of undertaking advocacy evaluation, mostly from the US perspective.
The authors consider advocacy evaluation more of a “craft” than an exact science:
“The real art of advocacy evaluation, which is beyond the reach of quantitative methods, is assessing influence..Advocacy evaluation is a craft—an exercise in trained judgment—one in which tacit knowledge, skill, and networks are more useful than the application of an all-purpose methodology.”
Advocacy impact Assessment Guidelines
Here is an interesting fact sheet from CABI.org – “Advocacy Impact Assessment Guidelines” (pdf).
The fact sheet provides a very good summary of evaluating advocacy actions – the “how” and “what” to evaluate. It also highlights some key points to keep in mind, summarised here:
- Different stakeholders will have different views on what success is;
- If you cannot prove impact, be satisfied with a critically informed assessment of change;
- Include subjective criteria, (i.e. what successes people feel have taken place but cannot substantiate with evidence);
- Break down your advocacy intervention into manageable components;
- Be practical, yet flexible. The external environment in which your advocacy takes place will be changing all the time;
- Monitor changes in your strategy itself;
- Collaborative advocacy means that individual contributions cannot be separated from the success of the whole effort;
- Share evaluation results with a wide range of people to show the disbelievers that advocacy can work and to motivate those who have been involved.
View the fact sheet here (pdf) >>
