Posts filed under ‘PR evaluation’
Barcelona Declaration of Measurement Principles – Final Version
The Barcelona Declaration of Measurement Principles for communication evaluation, which I wrote about previously, have now been finalised, here they are:
Seven Principles
1.Importance of Goal Setting and Measurement
2.Measuring the Effect on Outcomes is Preferred to Measuring Outputs
3.The Effect on Business Results Can and Should Be Measured Where Possible
4.Media Measurement Requires Quantity and Quality
5.AVEs are not the Value of Public Relations
6.Social Media Can and Should be Measured
7.Transparency and Replicability are Paramount to Sound Measurement
The explanatory text (pdf) is well worth a read as it explains the thinking behind the above principles. As I pointed out before, two out of seven principles focus on media measurement, indicating our sector’s focus in this area - what I consider a distraction from real “outcome” evaluation which communications needs.
The International Association for Measurement and Evaluation of Communication (AMEC) has set up taskforces to find the answers to two questions related to principles 5 & 6, notably:
1) What are the “validated metrics” to replace AVEs?
2) How do you get started in measuring social media, and what are the definitions of relevant metrics?
I look forward to learning more about their work.
More on the Barcelona Declaration of Research Principles
Further to my earlier post, you can read more about the Barcelona Declaration on a special page of the Institute for Public Relations (IPR) website.
The next step is that the principles will be refined based on feedback received during the Summit in Spain and additional input being gathered – you too can give an input, leave a comment here on the IPR page>>
First global standard of PR measurement
Yesterday, here at the AMEC summit, the first global standard of PR measurement, the “Barcelona Declaration of Research Principles” were adopted, as listed here:
Barcelona Declaration of Research Principles
1. Goal setting and measurement are fundamental aspects of any PR programmes.
2. Media measurement requires quantity and quality – clip cuts are generally meaningless.
3. Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity; they measure the cost of media space.
4. Social media can and should be measured.
5. Measuring outcomes is preferred to measuring media results.
6. Business results can and should be measured where possible.
7. Transparency and replicability are paramount to sound measurement.
Interesting to note that three of the principles mentions media measurement – reflecting the inbalance in the industry towards measuring media coverage. Hopeful ly this obsession with counting clips can be overcome and a move towards measuring outcomes is seen.
At the AMEC summit, the PR Society of America, presented its latest work on PR measurement, as you can see in the slides below. It’s well worth a look as it focuses on metrics for communications outcomes:
Web analytics and communications evaluation
When evaluating a communications project, I often consider the web metrics aspect of the project, if a website played an important part in the project. Web metrics are statistics generated by tools that measure website traffic, such as how many people visited a web page, where did they come from, etc.
Seth Duncan has recently produced for the US-based Institute for PR a very interesting paper on this subject:
The paper focuses on the aspect of referral (e.g. which is the most “efficient” source of traffic for a website) but also contains some intruiging descriptions of advanced statistical methods for web analytics.
Standardisation of PR Evaluation Metrics?
The UK government’s Central Office of Information (yes, I know it’s rather Orwellian sounding…) has produced a set of metrics for measuring PR campaigns: “Standardisation of PR Evaluation Metrics” (pdf).
Frankly, they are disappointing. The title is deceiving, it should be called “Standardisation of media monitoring metrics”. As that all the document covers – the superficial stuff – “reach”, “Favourability of coverage”, etc. These are “outputs” of PR activities.
But what about metrics for measuring “outcomes”? These don’t get a mention. Well there is an admission but you have to dig deep, they do say:
“It is worth bearing in mind that these standardised core metrics for media evaluation are only one component of any campaign evaluation. It is crucial to agree specific key performance indicators (KPIs) at the outset of a campaign.”
So they admit it, these are metrics for media coverage only. And no guidance is given on these KPIs (that are typically “outcome” level). If you are interested in learning more about metrics for “outcomes”, I’d recommend you start with the excellent guide from the Institute of PR: “Guidelines and Standards for Measuring the Effectiveness of PR Programs and Activities” (pdf).
AVE to WMC – A wolf in sheep’s clothes?
The Institute for Public Relations has published a new research paper explaining a new media measurement concept called “Weighted Media Cost”. But is this anything new – or simply the dreaded Ad Value Equivalent (AVE) in disguise – a wolf in sheep’s clothes? PR measurement guru KD Paine certainly thinks so.
I think that any measurement based on media space generated by PR efforts is bound to be flawed and increasingly illrelevant. Why?
- Generating media space is rapidly loosing importance as a PR objective – particularly with the growth of other ways that people can obtain information. These measurements typically look at print media – which is a media with a declining readership base
- Measuring how much media space was generated takes the focus away from the more important objectives to measure – what did PR efforts actually change in terms of knowledge, attitudes and behaviors of target audiences. That’s harder to do, but it’s worth the effort…!
Workshop on communications evaluation
I recently conducted a one day training workshop for the staff of Gellis Communications on communications evaluation. We looked at several aspects including:
- How to evaluate communication programmes, products and campaigns;
- How to use the “theory of change” concept;
- Methods specific to communication evaluation including expert reviews, network mapping and tracking mechanisms;
- Options for reporting evaluation findings;
- Case studies and examples on all of the above.
Gellis Communications and myself are happy to share the presentation slides used during the workshop – just see below (these were combined with practical exercises – write to me if you would like copies)
Award for Excellence in PR Measurement & Evaluation
If you are “into” PR/communication research and measurement, and have used it extensively in one of your PR campaigns this past year, you might want to consider entering the Jack Felton Golden Ruler Award for Excellence in PR Measurement & Evaluation. Entries of all types are welcome – including research using social media! The award recognizes superb examples of research used to support public relations practice. Winners are feted at the Institute for Public Relations Summit on Measurement in October in Portsmouth, NH, and it’s quite a big deal.
Be quick – Entries are due August 15th. Here’s How to Enter, and see these examples of previous winners’ entries: Padilla Speer Beardsley’s Winning Entry 2007 or Shell’s Award Winning Entry 2008 for ideas – and there are more on the site - also good if you want to see some best practices in communication / PR evaluation.
Is it possible to create a ROI on communication?
Is it possible to create a ROI on communication? Only if you pursue truly strategic goals…
Given the huge interest in a workable tool to measure ROI on communication, and the amounts of hot air generated on the subject in the two Berlins, I would like to propose a simple blueprint:
It goes like this:
Calculate the value of whatever it is you are trying to promote, sell or protect and the cost of failure.
From this you allocate a reasonable proportion as a budget, enough to enable success at selling, promoting or protecting, etc., recognising the price of failure. (In other words, if it’s not strategic or crucial, think about whether you really want to do it at all).
Measure your inputs – time, resources, materials.
Measure your outputs qualitatively (reach, tone, influence, prestige)
Measure your outputs quantitatively (impressions, views, footfall etc).
You may need to assign a value to your outputs, because PR is not the only game in town. Other communications disciplines will also have played their part.
Measure your outcomes. Not as tricky as it seems. Did you sell any more product? How many people came to the event? Did you save the chemical plant from closure? What was the value?
Don’t get hung up on opinion shifts, unless the shift contributes directly to the achievement of your strategic goals. If it does, then measure it.
I leave it to someone cleverer than me to calculate the golden ratios and percentages and a single number index that will make this work.
It may not be ROI, but it will be a measurement which could become meaningful.
I quote Katie Paine:
“If everyone is so darn hot on ROI, why aren’t they measuring it? I say the reason is, again, lack of knowledge about measurement and measurement tools. Because, there are measurement tools out there that will provide ROI (albeit, not simple or easy to use). So if they really, truly wanted ROI, then they would use measurement techniques that would allow them to calculate it. But instead, they focus on exposure-based methods like clippings and AVEs, because they are afraid of actually learning what the ROI of their programs is. They might find that their programs yielded considerably less R than the I that was put into them.”
I Agree. It’s because so many communications programmes do not have truly strategic objectives in the first place. “Raise awareness”, Increase visibility”, “improve customer feedback” are not strategic objectives, and it is impossible to calculate the value of success or the cost of failure.
Convince legislators on a particular course of action, convince a population that a plant should or should not be built, sell x million units of a new product, persuade us that a particular engineering material is non-carcinogenic and safe, or that a proprietary medicine works – now those are what I call strategic objectives.
They would all merit very grown up budgets, and it would be a dereliction of duty not to measure whether taxpayers or shareholders funds had been wisely spent.
Richard
