Eight challenges to measuring off site social media performance

Research Manager

Tagged with:

I’m an Audience Researcher within BBC Future Media, which means I contribute to my department’s remit to measure product performance, understand audiences’ attitudes and behaviour and provide recommendations to inform decision-making in the future.

One of the areas I work across is social media; an area that continues to grow in visibility and importance. As Holly Goodier blogged last year, 77% of the UK online population now actively participates on the internet using social tools.

The likes of Facebook and Twitter get a lot of attention, but social media extends beyond social networks to include functionality such as commenting or sharing that can make any website, including BBC.co.uk (and indeed this very page), social.

BBC ONE's Facebook page

However, for the purposes of this post, I will focus upon off-site activity – that which occurs outside of BBC.co.uk.

Off-site social media is a great way to reach and connect with our audiences, and as such we operate many accounts across our channels, brands and divisions to accomplish this. The majority of our accounts are within Facebook and Twitter, but several of our programmes and services also have a presence on sites such as Google+, Instagram, Pinterest, Tumblr and YouTube.

Watching our teams use these different environments in multiple ways is fascinating, but it also creates challenges when it comes to measuring how we are performing.

Listed below are eight challenges we face when assessing social activity, both directly related to our official presence and in terms of wider online conversations. Not all of these are limited to social media, and few if any are BBC specific, but they give an idea of some of the considerations we face:.

1. No official measurement source: TV has BARB and Radio has RAJAR – two well established bodies, with consensus on the most appropriate metrics to use. Within digital, there is the relatively new UKOM - while it offers a range of measures, it does not break down social media into specific accounts (such as @BBCSport on Twitter or BBC One on Facebook). Social networks may offer useful insight tools themselves, but only top-level information is made public. It can therefore be difficult to place performance in the context of the performance of other accounts or organisations.

2. Limited geographic restrictions: I work within the public sector side of the BBC, and so am principally interested in UK performance rather than global.  Again, insight tools can offer geographic splits but there isn’t much publicly available UK-specific data to compare to.

3. Aggregating across multiple accounts: It can be difficult to assess overall performance when multiple accounts are being used – for instance, if we wanted to measure combined performance across @BBCBreaking and @BBCNews on Twitter. Action-orientated metrics (such as measuring the number of ‘likes’ or views) can be added together, but others such as total audience cannot, since people that follow multiple accounts would be counted more than once unless data could be de-duplicated . The challenges of measuring your own organisation are magnified when trying to measure others.

4. Totalling activity across multiple services: The ideal would be to evaluate our performance across the entirety of social media, but different services with different functionalities with different ways of measuring make this impractical. For instance, is a Facebook share the equivalent of a Pinterest re-pin?

5. Distinguishing active from lifetime audience: Metrics such as followers or likes are based on lifetime activity – they take no account of recency and so could count activity from several years ago. Changes over time can be used to assess growth, but it doesn’t give an accurate reflection of the active audience – people that interacted with the site more recently (e.g. in the last week or last month). Again, some insight tools offer this function, but once again there is an inability to place performance in context.

6. Interpreting behaviour: Adding up the number of comments or mentions produces a measure of audience engagement, but it assumes all interactivity is good when in fact audiences could be using social media to protest against something or talk about how much they hate a particular programme. Sentiment analysis can provide some context. While tools continue to improve and innovate, ambiguities in tone and meaning mean that analysis is not yet fully accurate

7. Identifying relevant activity: Counting the volume of mentions for a programme across social media could be limited to searching by the programme name, or it could include a search for mentions of the on-air talent, topic or notable incidents. Furthermore, that on-air talent can appear across multiple programmes or formats. Agreeing on parameters can be hard to do. Some tools do automate this to provide a consistent view for all users, but without an industry standard it is still possible for other organisations to announce radically different figures due to different measurement criteria. 

8. Measuring impact: Metrics such as likes or retweets are not ends in themselves, but are signifiers of audience engagement. Social media objectives should be broader than stimulating this type of behaviour alone, and could have goals such as increasing the audience figures for a TV or Radio programme or raising positive opinion towards a programme, channel or service. This is something that is hard to measure in any medium, but the nature of social, where ease of interaction encourages high volume of messages – makes it harder than most to measure this type of impact.

In a forthcoming post, I will outline some notable performances within social media to date, the above challenges notwithstanding. In the meantime, feel free to participate below the line with your thoughts.

Simon Kendrick is a Research Manager, Audiences, BBC Future Media

Tagged with:


This entry is now closed for comments.

  • Comment number 8. Posted by Simon Kendrick

    on 10 Jul 2013 16:54

    Hello all, thank you for sharing your thoughts. This post is intended to be an introduction to social media measurement, and I aim to follow it up in future with others outlining specific performance metrics or case studies.

    @ MichaelBowers afraid I can't organise any competitions, but I'm interested in hearing about any successful (or unsuccessful) case studies that you know of. As you say, it is a great way to learn.

    @ Nu Studio I agree that for small and medium sized businesses then ROI has to be a major consideration, but outside of affiliate links it can be quite difficult to prove a causal link between social activity and an uplift in revenue. (Though of course this is assuming the return is financial, rather than other returns such as improved awareness or brand perceptions)

    @ Mike Downes I agree that social media can be a great place for employees of an organisation to engage and interact with audience members, customers or users - and many of our journalists and personalities do (for the record, my social media profiles are personal). I also agree that Google+ can be a great tool to use - and the BBC does use it, such as with https://plus.google.com/+BBCNews/ - but that whichever social services are used, the measurement challenges I outline above still apply.

    • This entry is now closed for comments. Number of positive ratings for comment 8: 0
    • This entry is now closed for comments. Number of negative ratings for comment 8: 0
  • Comment number 7. Posted by Mike Downes

    on 10 Jul 2013 10:09

    I've spent far too long reading this post and thinking as anything I write here (or at the BBC Internet Blog) will lead to little action.

    The way I think, I forget the BBC and look at the people, in this case +Simon Kendrick (who has a profile photo of Peter Faulk with last public entry July 2011) .. This tells me he has no interest or understanding of Google+ .. Now, some people may think I'm stepping over a line by looking this stuff up, I can get journalists and media people all over the globe, sure you can yourself.. if only they show themselves..

    What I'm getting at, the way you are dealing with the BBC and not people. I had to raise a formal complaint to the BBC before I had a personal email, even then on my reply I was greeted with auto out office reply.. Googlers never do this this, we operate an always on mentality..

    The picture I'm painting here, the way this blog post should be written is by stepping off the BBC ship and holing up in a cafe somewhere and watching the world float by.

    You may then see people in the UK use Google Search 90.55% of the time - with that comes Google Maps, YouTube, Gmail, Google News and a bunch of other services that, wait.. are connected by Google+ ..

    So when I saw the BBC Twitter Egyptian Translation post (mentioning Microsoft/Bing and then facebook, but not Google) I thought something was wrong. https://plus.google.com/+MikeDownes/posts/41YUcshQemt

    From my formal complaint, I got the stock,

    ''Thanks for bringing this to my attention. The article now refers to Google.
    However, I would note that the story focused on social networks - and while Google has one it is less widely used than Twitter and Facebook, which was why it was not initially included.
    I hope this addresses your concerns''.

    I replied immediately with this text and many links:

    ''While I appreciate the article now includes Google - your team need to update your opinion on Social Networks. Google+ has more active users and therefore is bigger than Twitter. I can give you all the data you need on that.. Google+ 359 million active users, Twitter 200 million..''

    Here's the killer punch, I sent a second email after waiting 5 days and got this:

    ''The issue is that all Google accounts are Google+ enabled by default these days. As a result many of these surveys class someone as being an active Google+ user even if they never actually visit the site'..

    ''.. Having said that we do report on the topic when we deem it news worthy. In some cases we don’t cover it simply because of a lack of information – for instance yesterday we asked Google+ for data on Andrew Murray’s Wimbledon victory but the firm said it had nothing to share beyond the number of searches''

    In conclusion There are more that eight challenges that face the BBC and how the people use the internet..

    If 90.55% of the UK start by searching for stuff in Google - then why do the BBC not embrace that and get stuck in and committed to what the people do ie GO GOOGLE ..

    Video on Mobile is HUGE, YouTube has a billion users - and guess what it's connected to Google+ .. Chrome has 750 million users, Gmail 425 million users..

    My message, put the politics aside and get Google. On 31 March 2011, the BBC reached out meet local bloggers. They wanted to learn. That seems to have stopped. http://www.whatsinkenilworth.com/2011/03/bbc-meet-hyperlocals-from-west-midlands.html (BBC's +matthew eltringham and +Laura Ellis were there) ..

    I was once interviewed by BBCCoJo for 90 mins, it started with, ''How come you can work faster and cleaner than the BBC?'' .. I was like, ''Is that what you are saying?''.. they were like Oh yeah ..

    So, +Simon Kendrick, if you want to measure off site performance, get in touch with the people who use the fastest and largest social media networks on the planet. Hangouts were not around in March 2011 - they are now..

    Failing that, get the BBC to open their doors wide to people like me and start listening.

    • This entry is now closed for comments. Number of positive ratings for comment 7: 0
    • This entry is now closed for comments. Number of negative ratings for comment 7: 0
  • Comment number 6. Posted by Nu Studio

    on 10 Jul 2013 08:25

    Social Media is the most hyped up marketing tool ever in my opinion. For large scale organisations like the BBC it makes a lot of sense to have a good and active presence on social media. They also have a budget larger than small countries and it is well spent on social media. However for the average business social media is a joke. They are spending more time and more money on this form of marketing, and with no real way to measure the ROI they are happy to keep spending more money on it. It's a shame really. [Unsuitable/Broken URL removed by Moderator]

    • This entry is now closed for comments. Number of positive ratings for comment 6: 0
    • This entry is now closed for comments. Number of negative ratings for comment 6: 0
  • Comment number 5. Posted by MichaelBlowers

    on 9 Jul 2013 10:57

    The comparison of results across the broadening range of social media outlets is becoming a major concern to big brands and it is understandable that many view them in isolation, creating their own objectives and strategies.

    I applaud the BBC for opening up the discussion because social is not going to go away and content is not going to fall. The solutions will come from good case studies which transparently discuss the learning process, illustrate the pit-falls and what might be the look of success; these are priceless!

    Maybe you should hold a competition for people to suggest the best ways to measure/add insight?

    • This entry is now closed for comments. Number of positive ratings for comment 5: 0
    • This entry is now closed for comments. Number of negative ratings for comment 5: 0
  • Comment number 4. Posted by Pez

    on 9 Jul 2013 09:14

    I really appreciate your post and you explain each and every point very well.Thanks for sharing this information.And I'll love to read your next post too.
    PET preform

    • This entry is now closed for comments. Number of positive ratings for comment 4: 0
    • This entry is now closed for comments. Number of negative ratings for comment 4: 0
  • Comment number 3. Posted by Simon Kendrick

    on 4 Jul 2013 13:00

    Hello - and thank you for your comments

    @ Craig Smith - I do agree that different social services can excel in different ways, and we do aim to utilise their relative strengths - hence why some of our programmes, stations or channels have a presence on multiple services. One of the challenges we have in summarising the performance of our accounts or campaigns is balancing the nuance of performance across a variety of channels with an ability to succinctly and directly assess the performance at a total level.

    @ Michelle Brown - you make a good point about the connectedness, and how we aim for the "whole" to be greater than the sum of its parts. We're always thinking about the best performance metrics, and I hope to continue to share our thinking on this blog.

    • This entry is now closed for comments. Number of positive ratings for comment 3: 0
    • This entry is now closed for comments. Number of negative ratings for comment 3: 0
  • Comment number 2. Posted by Michelle Brown

    on 4 Jul 2013 12:10

    @1. Craig Smith - I agree it's difficult but I think the performance is less about the type of media and more about the "social connectedness" of the content itself. The performance answer does not lie in simply aggregating the performance across the different applications - it's more likely a practical application of something like Tony Hirst's social mapping approach - see http://www.bbc.co.uk/blogs/blogcollegeofjournalism/posts/How-to-map-your-social-network

    Understanding how content is referred to and linked to across social media apps gives a stronger indication of its importance and relevance. Perhaps the BBC can put forward some thought leadership around social mapping-based performance metrics... I am certain that it be a valuable piece of work!

    • This entry is now closed for comments. Number of positive ratings for comment 2: 0
    • This entry is now closed for comments. Number of negative ratings for comment 2: 0
  • Comment number 1. Posted by Craig Smith

    on 4 Jul 2013 11:58

    The importance of social media with respect to the news industry is without doubt increasing and there are plenty of articles about that like http://sizwemahlala.blogspot.com/2011/11/social-media-and-future-newsrooms.html about it. Measuring the performance of, and indeed the return on investment in social media initiatives has proven notoriously difficult though.

    I think it would be difficult to sensibly consolidate reporting on social media activity across all applications (by applications I mean Facebook, Twitter, Pinterest, YouTube etc).. each social media outlet has a different focus and different sorts of content gravitate towards that application. E.g. YouTube is likely to be sharing medium of choice for video but not necessarily so for a quick snippet on breaking news. Reporting social media performance by application would make much more sense.

    • This entry is now closed for comments. Number of positive ratings for comment 1: 0
    • This entry is now closed for comments. Number of negative ratings for comment 1: 0

More Posts