Measuring the influence of Twitter
As Twitter becomes more present in communications, is there any way to measure how influential it is? Well, there are plenty of tools to monitor Twitter usage. But the MetricsMan blog has some wise word of caution about these tools. He warns that the tools are not really measuring influence, as he puts it well:
The problem here is no one is actually measuring true Influence – the ability of one individual to change another’s opinions, attitudes or behavior. You can’t surmise whether or not an opinion or attitude has been impacted, you have to conduct research. Opinions and attitudes exist within individuals. You cannot assess this by proxy, looking strictly at online metrics. Online behavior can be measured without primary research, but offline behaviors have to be observed or reported.
Add comment February 7, 2010
AVE to WMC – A wolf in sheep’s clothes?
The Institute for Public Relations has published a new research paper explaining a new media measurement concept called “Weighted Media Cost”. But is this anything new – or simply the dreaded Ad Value Equivalent (AVE) in disguise – a wolf in sheep’s clothes? PR measurement guru KD Paine certainly thinks so.
I think that any measurement based on media space generated by PR efforts is bound to be flawed and increasingly illrelevant. Why?
- Generating media space is rapidly loosing importance as a PR objective – particularly with the growth of other ways that people can obtain information. These measurements typically look at print media – which is a media with a declining readership base
- Measuring how much media space was generated takes the focus away from the more important objectives to measure – what did PR efforts actually change in terms of knowledge, attitudes and behaviors of target audiences. That’s harder to do, but it’s worth the effort…!
2 comments January 29, 2010
Presenting evaluation results – interactive mapping
One of the challenges faced in evaluation is presenting evaluation findings in way that facilitates their use, as I’ve written about before.
Now here’s an interesting idea – presenting evaluation results in an interactive map. This example is for the monitoring, evaluation, and communications for an agriculture development program in Afghanistan. Here is a screenshot of the map:
Interactive map produced by Jasha Levenson of Cartametrix.
Add comment January 21, 2010
20 minute webinars for evaluators
The American Evaluation Association is presenting a series of 20 minute webinar-based demonstrations by and for evaluators – Demonstrations are free for members of the AEA and non-members can purchase a demonstration pass ($80 for unlimited demonstrations for one year, $30 for students including AEA membership).
Here are the next scheduled webinars:
Thursday, February 4, 2:00-2:20 PM EST: Submitting an AEA Conference Proposal - Susan Kistler
Tuesday, February 9, 10:00-10:20 AM EST: Tools for Evaluation Teaching and Training: Approaches to Introducing the Concept of Evaluation - José María Díaz Puente and Michael Newman
Thursday, February 18, 2:00-2:20 PM EST: Using Jing for Teaching and Learning – Theresa Murphrey
Add comment January 13, 2010
New resources on advocacy evaluation
Here are some new resources on evaluating advocacy campaigns:
Practical Guide to Advocacy Evaluation from Innovation Network (pdf)>>
For further resources and information on advocacy and campaign evaluation, please consult past posts on these subjects.
Add comment January 9, 2010
Online training course on results-based Monitoring and Evaluation
Here’s an interesting online course offered five times in 2010:
“Results-based Monitoring and Evaluation
This 10 hour live online course will enable you to prepare the ground for undertaking monitoring and evaluation and analysing the results. Results Based Monitoring and Evaluation will help participants understand the key components for effective M&E in order to improve management performance and evidenced based decision making. The course explores how to establish the necessary framework for good M&E based on clarifying stakeholders, bringing the Logical Framework up to date, and generating a set of criteria against which to measure progress.”
Note: this blog has no commercial association with the course organisers – it just seems an interesting course!
Add comment January 3, 2010
A new evaluation method: The Evaluation Café
We are always on the lookout for different methods and approaches for evaluation. Here is a new method that we haven’t come across before: “the evaluation café“.
Following is a brief description:
The Evaluation Café is a method for group facilitation that allows stakeholders of a project or programme to evaluate its impact in an informal brief session. The purpose of the Evaluation Café is to build and document stakeholders’ views on success and impacts after a planned activity.
Add comment December 19, 2009
More resources on network mapping
As I’ve written about before, I’m very interested in how network mapping can be used in evaluation.
Here are two excellent resources for people wanting to learn more about this research technique:
1) A training course conducted by Steeve Ebener of WHO on “Social Network Analysis, Mapping social relations”. You can view the training slides for eight sessions – and there is really some excellent examples of how network mapping can be used.
2) A manual “Network Mapping as a Diagnostic Tool” by Louise Clark (pdf). A “how to” guide on network mapping and an explanation of how to use a popular network mapping software UCINET (I use it too, it’s the best I’ve found).
Add comment December 11, 2009
Evaluating Strategic Communication for Avian Influenza/Pandemic Influenza
As the world is focused on the current flu pandemics, we have seen many efforts to communicate prevention and treatment approaches.
And what about how to measure the results of such communication efforts? Here is an interesting set of guidelines from UNICEF on this issue:
Although it’s a technical document, it provides interesting insight into sampling and interviewing techniques for evaluating communication campaigns.
Add comment December 2, 2009
Summarizing evaluation reports
As I’ve written about previously, evaluation reports are notoriously under-read and underutilized. Aside from the executive summary, evaluators need to find ways of presenting their key findings in a summarized format that make them attractive to their publics.
Aside from the predictable Powerpoint summary (which still can serve a purpose), some of the techniques I have used – and that were well received by publics – are as follows:
Multimedia video: using interviews, graphs and quotes in a video to bring the evaluation results “to life” (see this post for an example)
Scorecard or “snapshot”: highlighting the key findings graphically in one page. See this example:
Summary sheet: summarizing the main findings, conclusions and recommendations in fact sheet of 2-4 pages. See this example: Summary Sheet (pdf)
Findings table: summarizing the main findings, particularly useful where the evaluation is responding to pre-set objectives and indicators, as per this example:
I’m always interested to learn of new methods to summarize evaluation findings, so if you have any more ideas, please share them!
Add comment November 25, 2009
