Debbie Klein: We’re going to out beard and out cat everyone tonight, because we have got the ‘Oh My God’ cat off YouTube, which, if you haven’t seen it it’s had over 20 million views, and there are some amazing spoofs off Justin Bieber for example doing that.  Literally like that! No social media presentation would be complete without a cat, no doubt! The thing that senior clients ask me most often is ‘what are other clients spending their money on, other than traditional media?’ What’s growing across the Engine family.  Is it data? Is it sponsorship? Frankly if sponsorship isn’t growing this year then God help us all! But the answer is, I don’t even have to speak to anyone to know, I just have to look on the second floor of the building, because these guys Jam, who are our social media and mobile agency, have (social media) has quite eaten my desk because I’ve had to leave where I was sitting because they’ve grown so quickly.  In the last two years, they’ve gone from 22 people to 85.  And that’s because clients have seen that anyone with a keyboard and an iPhone can become a broadcaster and can have an opinion.  And as soon as they’ve worked that out, the second question they as it ‘so, why should we do it? And how does it work? And they very quickly get onto ‘and is my money working?’

Debbie Klein: So if you look at the last 3 years and the interest in this area, I mean social media as a concept has been around since 2004, but it’s really only in the last 3 years (if you look at the Google search stats since 2008) the concepts of social media measurement and social media ROI have even come into being.  So it’s brand new and it’s really in the last 12 months that clients have been really demanding, quite rightly, proof of how this is working, and asking us to help guide them in how they should measure it.  So I then come back from those meetings and talk to Jamie, who’s incredibly well placed to advise them because he grew up in traditional advertising, then worked in a media agency, and is now in social media, and say ‘we need to have some framework’.  So that’s what we’ve been working on, and have put a lot of resource behind it to try and give us both really a useful point of view about how we can work out which half is working in the old phrase, but also advise clients on why and how they should carry on spending their money.

Jamie Kenny: Thank you.  So I think it’s really worth just starting with some context.  We all know social media is massive, so Facebook allegedly hit a billion users in September and there’s tonnes and tonnes of stuff on social media, on social networks.  This is one of my favourite posts.  The kid says: “Dad’s on Facebook WTF?” The father says: “Henry what’s WTF?” “Welcome to Facebook Dad” says the kid.  And that kind of sums it up for me.  There’s some of us who maybe don’t understand it as well as others, but that’s rapidly changing and I think measurement, how we need to approach measurement as industry is really growing up, and growing up at rapid speed.  So it’s worth sort of saying, a couple of years ago, and for a long time, the focus has always been about friends and fans and followers.  This was Starbucks, the first brand to reach 10 million followers on Facebook in July 2010, and there was a big fanfare.  For brand owners and for marketeers, fans are quite a new thing.  We could see, or we thought that people genuinely liked our brands and that fans and the whole follower thing and the number of followers has been a really big thing around social.  And people have said ‘we’ve got lots of fans’ and there’s kind of a fan grab.  Starbucks now have around 41 million, Coke probably the biggest Facebook following has about 43 million followers on Facebook.  But what I want to say is it’s completely not about the number of fans or followers that you’ve got.  The value of a fan if they do nothing…is nothing.  And I think that’s really important to bear in mind in where we’re going these days.  Because there are really some big established and fast-growing networks.  So Facebook in the UK, 26 million users.  Most of you will all be familiar with those big networks.  Twitter, 6 million users.  Interstingly MySpace was really big, 18 million a few years ago, is slowly coming back.  But equally there are the new kids, the relatively new kids on the block.  So Tumblir, 3 million users.  Instagram, you all would have heard about Instagram.  I was listening to the today programme talking about Instagram and why Facebook bought Instragram for a billion dollars, if that goes through.  They’ve got about 50 million users globally, about 500 million photographs on it, and about half a million users in the UK.  I’ve no idea why…I mean that evaluation is just beyond the payoff.

Jamie Kenny: But there are other areas and networks that are also growing fast.  Pinterest…If you’re doing up a house, or the cliché is if you’re having a wedding, you’ll be on PinterestDraw Something is probably the new starter.  So Draw Something has been going for about…well it’s come into the market very recently and they’re putting on about a million users a day, so they got to the first million in 9 days.  They’re at 50milliion at the moment, that’s the last stat.  They were bought by Zynga for 200million and allegedly they’re generating about $200,000 of ad revenue every single day.  So if I was Zucheberg, I would have brought Draw Something rather than Tumblir, but I’m sure he’s got a plan! The point is that there are lots of networks and those networks are becoming new, and there are lots and lots of opportunities, but equally there is just a huge amount of data.  This is from a Facebook brand page.  I’m sure lots of you will have brand pages, but when you take the data from your brand page and you export it into Excel, you get 65 tabs of data showing the performance of your brand page.  When you look at your post-performance, you get another 7 tabs of data.   That data is being updated every single day.  So there’s a huge amount of data to deal with.  If you’re thinking about measuring things in Twitter, you can look at the Twitter content, you can look at the number of followers, you can look at who they’re following, you can look at app mentions, you can look at peer index scores, cloud scores…the list goes on.  This is data puke and this presents a real challenge for brands and for agencies.  I think the challenge is what do you measure? What do you look at? What’s important? And we spend a reasonable amount of time trying to figure this out, and trying to figure this out with the clients that we work with.  The first way we do this, is we sit down.  We obviously understand the client’s business objectives, their marketing objectives and their social media objectives, but then we list out all the various different data points and we map them out and we think about what’s going to be valuable, what’s going to be less valuable, what’s going to affect it in the short term, and what’s going to drive business in the long term.  And it is going to be different for lots of different brands, but the point is you need to sit down and think about, and list out, and get all of those data points.  And ultimately, you need to apply a very simple question which is the so what test.  What does it mean for me? What does it mean for our business? Because I’d argue stats, like followers, potentially, certainly like friends of followers, which is a big Facebook metric, are pretty worthless.  Certainly following to follower’s ratio, again, pretty useless!

Jamie Kenny: So we sit down and we figure out what works and we always apply that so what test.  By doing this, we spend a bit of time and we’ve developed what we call the measurement score card.  And it basically covers six areas, and I want to talk through each of these six areas.  The important thing to say about the measurement score card is it’s actually focused on participation.  It genuinely looks at what people are doing, and it looks at and is focused on, what happens after you do a post, a tweet.  And that’s really important.  We’ve applied that so what test and we’re looking at those things.  Equally, and this is fundamental, is that it applies to whatever social network you are on.  So whether I’m on Facebook, whether I’ve got a page on Pinterest or Twitter, this framework works across those.  There are two areas on the framework, and the first is around engagement.  We really focus on, not followers, but actually what those people do.  And we’ve broken it down into three areas.  The first is the applause rate, so this is the rate that content is endorsed by users, by the audience.  So that might be, on Facebook, the number of likes.  Or on Google+,  plus ones, or on Twitter, favourites.  And on Twitter not many people will favourite tweets, but it’s important because people are endorsing that content.

Jamie Kenny: The second area is application, so this is really simple, the rate at which your content is being shared across the social graph, across the social web.  So the number of re-tweets, the number of shares per post in Facebook.  And the last area is around conversation, and this really looks at the rate at which you’re engaging with your audience, and your audience are engaging with you.  So the number of comments on your post, the number of app replies, they’re all really focused on engagement and what actual people are doing.  And that means that actually some of the numbers on these rates and going to be really small compared to the 40 million likes, and that’s important as well.  Beyond engagement, we look at sentiment.  So I’m sure a lot of you guys have heard off buzz monitoring.  So buzz monitoring is technology, software that listens to conversations about your brands, about products, about markets on the web.  And lots of companies like Radience 6, like Brandwatch and there are lots of new software providers providing great tools, and they genuinely are great tools and they can show you the volume, and they can show you where those posts are happening.  And to give you an example, for Renault there are probably about 500 posts a week.  For Sky, there are probably about 7,000 a day.  So there is a vast difference depending on your brand.  But one of the things the tool looks out is sentiment.  So is your post positive, neutral, or negative.  And all these software companies claim with automated sentiment that it’s really important.  The problem is that the automated sentiment doesn’t really show the true picture.

Jamie Kenny: So what we do is we track sentiment over time, and we track it as a net sentiment score, some of you will be familiar with a net promoter score, so we look at the positive versus the negative.  We exclude the neutral because often neutral is huge amounts of spam or news.  And we track that over time, and interestingly you can see, depending on what you’re doing, depending on your activity, how you improve that sentiment over time.  So for some markets, like the broadband market, sentiment is really negative.  If your broadband breaks, one of the first things you go and do is you go and tweet and you complain like hell about it.  How you get online is a good question, you use your mobile or you go to work, but sentiment is really negative.  Within the automotive sector, on the whole, sentiment is higher and one of the things working with Renault is, we’ve been able to shift that sentiment over time.  So we look at about 500 posts a week, we probably respond to about 2.  There are about 20 to 30 per day that are important, that we’re flagging to the clients, and we’ve developed our own system called jam track, our own dashboard that sits on their mobile, sits on their tablets and is over the web.  But it flags the important posts, because lots of them aren’t important.  And importantly is we’re looking and we’re giving them a manual sentiment score.

Jamie Kenny: So this is a test that we did, looking at a huge amount of data for one of our clients, and we put the best of the machines, the best of the software up against human analysis.  So I think there was about 5 or 6 thousand posts.  We got one of our skinny jean wearing people to go through, and it wasn’t particularly interesting, but it just showed how wrong the machines are.  So you can see negative sentiment from the machine shows there were 7% negative, but actually when we did the proper analysis, 66% was negative sentiment.  There’s a huge issue there, and if you pay attention to automated sentiment, you’re going to get readings like this, because automated sentiment isn’t good enough.  And I think it’s really important, there’ a challenge for businesses about the resource it takes.  If you’re Sky, that’s a huge amount of resource, and you have to find ways to look at that.  But would you necessarily automate your brand tracker or your PMS? You wouldn’t leave it to a machine to do that.  So for the moment, we’d absolutely advocate looking at manual sentiment, and looking at the net sentiment score over time.  So that’s the fourth area.

Jamie Kenny: The fifth is really about delivering media efficiency.  Lots of people talk a lot about social media as being cheap.  It’s kind of free media.  Clearly, it’s not cheap.  There’s a cost, there’s a human cost to that.  But one of the ways we look at and judge the effectiveness of social media is looking at the equivalent cost.  So particularly looking at the cost of that earnt media.  I’m just going to play you a quick video that will demonstrate a campaign we did, and I’ll show you.  It basically ran in about 15 train stations over about 5 days.  Normally you would buy the out of home, but what we did is we integrated it with Twitter.  So people would tweet the billboards, we had a team that would look at their profile, look at how influential they were, and look at what was interesting to them.  Then we would tweet them back, via the billboard, and make recommendations about what they should watch on TV that night.  The interesting thing is how do you measure that? How do you give that a value? So what we looked at is, ok if we’d just done the billboard, if we’d just done a standard execution, the cost per viewer would have been about £2.15, but by adding in Twitter (it gave us a huge amount of increased coverage) so just out of home would have had a coverage of around 71,000.  BY adding in Twitter, because people then shared it, we got coverage of around 985 thousand.  So cost per viewer at 18 pence.  Now granted, the impressions on Twitter are different from the impressions on a billboard, but that is just one way, in terms of really trying to understand the value of earnt media.

Jamie Kenny: The last area in terms of the scorecard is driving action.  Clearly, what you have to do within social, and ultimately we want people to do certain things, and those things will vary.  They will be visit a website, they will be buy a product, or in the case of Samsung, we wanted people to watch a product review.  So when Samsung launched their new GS2 phone, one of the things we thought about was how do we get people to watch that product review in an interesting way? And if you’re familiar with tech and electronics, one of the weird internet genres at the moment in unboxing videos.  So tech-bloggers will get a new product in its box and they will unbox it and do a review of that product, be it TV or a mobile phone.  And these products are really popular because they explain all the features.  So for Samsung we thought actually how do we make that more interesting? How do really get our message across but mash up the unboxing genres? So we came up with the idea of extreme unboxing.  We took six of the biggest tech bloggers from the nex web and the like and we said ‘we want you to do an unboxing video for us, but we want you to it in a different way.’

(Video plays)

Jamie Kenny: The purpose is, so he’s one of the pre-eminent tech bloggers.  That video was hosted on the nex web, also hosted on Samsung’s YouTube page.  When we looked at what it did in terms of ultimate performance, clearly it drove a lot of views.  We had about a million views over four weeks across six videos, and if you look at the applause rate, the amplification rate, and particularly the conversation rate, they were really high.  So particularly the conversation rate, around about 5,400.  That is phenomenal.  I think there were about 23,000 tweets alone, just about those videos, because they were really just built for social, and using people who were pre-eminent on the web.  Scorecards are measured on engagement and value.

Jamie Kenny: A couple of other things.  One is benchmarking.  You need to figure a way to benchmark your performance, and it’s enormously difficult.  People are getting a lot better at it, and there are benchmarks.  If you look at Twitter, Twitter is one of the areas all about speed.  So the half-life Bitly (a kind of player in terms of links), they studies the halflife of a tweet and it’s about 2.8 hours.  So after 2.8 hours, half of them have clicked on that link.  So Twitter is all about speed, so you need to look at your tweets and see how long people are engaging with those tweets, and how long after the posts they’re clicking.  Equally, someone talked about customer service via Twitter, and this is a huge area.  If you looked at the number of brands in terms of how they’re delivering, how fast they’re responding on Twitter, so the real people who are excellent at this are doing it in an hour.  And people (consumers) are increasingly expecting that response time.  So the likes of Orange in the Uk, Sky are excellent at it and particularly within the stage people like ComcastZappos.  You tweet them, and you’ll get a response in under ten minutes around customer service issues.  Facebook, there are a couple of important stats to bear in mind about Facebook in terms of benchmarking.  One is that the average status update is only seen by 16% of your audience, and that’s really important.  You need to check with your agencies, with the people who are managing your communities, are you exceeding that average? And if not, why not? And devise a content strategy to look to increase that.  Equally the talked about rate is between 1% and 2% and that’s really important to focus on, because it ultimately affects the edge rank.

Jamie Kenny: A couple of quick things to finally say.  One is the whole idea of value of fans.  The value of the fan, and there have been some studies…one study said it was $3.60, another said $136…absolute nonsense! Don’t pay any attention to the value of the fan.  The value of the fan will absolutely differ completely for all your different brands; it will completely differ between each and every fan.  And there is going to be a direct value in they are going to buy stuff from you, and there is an indirect value if they are influencing people.  But the value of the fan debate, we tend to ignore.  Equally, when it comes to the elephant in the room, what’s the ROI of social? Well it really depends, and it depends ultimately on how you ask the question, I think.  Because if you’re interested in, and you’re providing your agency with real business objectives, and those objectives are about driving financial value, then yes, absolutely, you can figure out the ROI.  But if you’re focused on increasing, and your objectives are about increasing buzz or gaining customer insight, or improving brand perception, then you can’t figure out the ROI.  SO we increasingly work with brands, and one brand we’re working with (Tesco mobile) we really focus on, and get them to give us the business objectives, because that way you can figure out the ROI.  You need to look at it over time, and you need to look at it and be very specific over it.  So this is one client that we work with, this is their Facebook performance over the last year, they’ve grown their fans, but importantly they’ve driven over £4 million direct revue from Facebook.  They tracked that based on the last click.  So people are in Facebook, and they’re going to their shop online, and were able to track it.  So it doesn’t look at the full attribution.  What we’ve been able to show is a really clear ROI.  We’ve looked at the return, which is just over £4 million, and we’ve looked at the costs.  So agency costs, production costs, there is a small amount of media costs in there, and also the likes of their client salary costs, those people who are responsible for managing that page, we’ve been able to show an ROI of £5.80 for every pound that they invest.

Jamie Kenny: So I’ve probably talked a lot and gone a bit slower than I’d hoped! The ICYMI is, in case you missed it, and really to try and re-cap is to try and move beyond focusing on the number of fans.  I suspect most of you have.  Really think about engagement, think about business value, seek out some of those benchmarks (and as an industry there are lots, and they’re getting better and more detailed).  Ignore some of that value of fan hype, but set business objectives.  Really important! Give your social media agency business objectives, and then share your sales data with them, share your CRM data.  Because ultimately this is really moving into the area of data marketing, and we talk about social as being brand performance marketing.  To get the best out of your agency, you need to do those things, and the last thing is what we call firing cannonballs and bullets.  And bullets are lots of small, cheap things, and sometimes they miss the target completely, and sometimes they hit the target and when they hit the target the target, we follow it up with a cannonball.  So it’s test and learn, and that approach is really important for social.

Hilary Cross: I think there’s so much food for thought for me there that I’ve just got to go away and think about lots of things, about what we need to do.  But I do recognise that we need to be so much more sophisticated than just likes and followers and those kinds of things.

Giles Pearman: We use social media as a vehicle to engage those audiences in a powerful way, and we get a lot of attraction as a result of doing that.

Chris Sorek: The interesting thing about what I see with what you’re talking about, definitely on the Facebook side of it, is having really rated whether or not somebody that’s on Facebook really accounts for anything.