social software measurement as asymmetric warfare

A couple of weeks ago, I was at the Sydney Social Media Club where there was much discussion of measurement by Paull, Jye & Matt.

After the event, I was pondering this well-known story:

Two campers are walking through the forest when they suddenly encounter a grizzly bear. The bear rears up on his hind legs and lets out a terrifying roar. Both campers are frozen in their tracks. The first camper whispers, "I'm sure glad I wore my running shoes today." "It doesn't matter what kind of shoes you're wearing, you're not gonna outrun that bear," replies the second. "I don't have to outrun the bear, I just have to outrun YOU," he answers.

When you are asked to demonstrate the ROI of social software, all you have to do is prove that SM delivers the benefits of traditional media at a lower cost - because it seems that most marketing is a cost of doing business rather than a generator of sustainable growth.

There's a problem here however: What if it doesn't? If you compare Twitter to TV using TV's metrics then TV is probably going to win. TV's metrics have coevolved with the medium over decades. It's a fixed match. TV is wearing the running shoes and you're looking like a tasty snack for a Mr Grizzly. Julian does some very interesting crunching of Twitter reach numbers and concludes: Knowing this is great for planning but who is going to take back a reduced number when traditional media agencies are still talking inflated reach into the 100,000s?

In making this move, I would suggest that the SM crowd risk doing a Mullah Nasrudin:

Nasrudin found a weary falcon sitting one day on his window-sill. He had never seen a bird like this before. "You poor thing", he said, "how ever were you allowed to get into this state?" He clipped the falcon’s talons and cut its beak straight, and trimmed its feathers. "Now you look more like a bird", said Nasrudin.*

Even by calling it "social media", we clip its talons. As this article by Malcolm Gladwell explains, if you are a David and you play by Goliath's rules then you then he will squash you. If you are engaged in an asymmetric struggle then you need to choose your turf and your weapons - not let them be chosen for you.

Some hypotheses (which may well be wrong because I don't work in marketing):

• Social software is that it allows you to do multiple things at once - customer service AND promotion AND sales AND research. Organisation with a combined approach to selling, marketing & service would be more open to social software plays that offer an integrated approach to the customer (not that are necessarily many of those about).
  
• Social software is speed. Measures around "time-to-market" and "response time" become critical here (which are more associated with new product development & customer service than advertising).
• Social software is cheap (unless someone is greedy) - "cost per message" anyone? This means that you can run multiple experiments in a short period of time at low cost. Iterating TV spots is not cheap.
• Social software is interactive - you'd expect that cross-selling/up-selling metrics might be relevant here.
  

Of course, these measures don't help you persuade people that Twitter is better at TV's job than TV is - but should you be trying to do that in the first place?

There are two other riffs in my head:

• Marketing & media as ecology (with prey, predators, parasites, symbionts).
• Measurement as social proof.

But those will be other blog posts.

*Thanks to Dave for introducing me to this story.

meauring organisational health

The following was in answer to a question posted to the AFN email list. Several people liked it so I thought I share it more widely.

"I'm a bit obsessed with measurement at the moment so here's my 2 cents. Depending on what you mean by "health & wellness" (physical, mental, emotional, social), you have a whole range of options.

They probably fall into four buckets:
- Existing operational HR data (as noted by Belinda) - staff retention, sick days,
- Direct surveys aimed at aggregating individual experiences (as noted by Belinda, Hans & Cory) such as staff satisfaction & the human synergistics stuff.
- Direct surveys aimed mapping connections between individuals (social network analysis) - see here for an example: http://www.robcross.org/pdf/roundtable/energy_and_innovation.pdf
- Indirect surveys using narrative (as noted by Mary Alice). These can either be done using F2F workshops or online software. This can yield both qualitative & quantitative insights.

A fifth area might be some kind of ethnographic research (i.e. getting stuck in there and having a look around) - but this yields qualitative rather than quantitative insights.

You probably want some combination of the above depending on the organisation's:
- Budget
- Culture
- Previous experiences with these approaches."

social network analysis fun & games (2)

Krackhardt's Graph Theoretical Dimensions of Hierarchy; Density; E-I index; Cliques; N-cliques; N-clans; K-plexes; K-cores; F-groups; Lambda sets and bridges; Automorphic equivalence; Optimization by Tabu search; Two-mode SVD analysis...

The list goes on. SNA has its own arcane set of techniques and an equally arcane language to decribe them. Other forms of data mining have their own obscure vocabularies (CHAID, CART, k-Nearest-Neighbour) and they have been heavily used in the CRM world. As we have tended to think of customers as isolated actors, this kinda made sense - but if that assumption was once correct (and I doubt it was) then it is decreasingly true.

So can we use N-clans & E-I indexes to make more money? I mean sure we want to engage with customers, improve service, yadda yadda yadda but can they earn us cold, hard cash?

measurement in complex systems

Further to the previous post, why do we measure? Operationally the main reason we need information to make decisions (indeed to find out if we need to make a decision in the first place). Quantitative information is needed to work out how much of something we need to do.

Now in ordered systems, that's straight-forward. My car is 2 litres low on oil. So I add 2 litres of oil. Job done.

The issue is that many systems are not ordered and these complex systems have 2 annoying properties:

1. Input-output may not be linear. A small change to the system may have a big (even catastrophic) effect.
2. By measuring the system we may actually change its state - e.g. sending out an employee engagement survey may actually raise or lower employee engagement.

So these 2 properties require us to:

1. Measure the system more regularly when we make changes so we can understand whether our impact is greater or smaller than expected. We need to sense it.
2. Use plenty of indirect measures that are less likely to be disturbed by our intervention.

Now I believe that the social media ecosystem is more often a complex environment than it is a simple one. So we need measurement systems that fulfil the 2 criteria above.

A little

Often

And off to one side.

Let me go off and find some examples of what these might be. One example might be the monitoring of tag clouds.

how long is a piece of string? (and who does it join?)

So the recent posts by Gav & Katie have mingled in my brain with the SNA work I have been doing.

Trad market research is based demographics. Although it lumps people together in groups (by age, location, wealth, gender, race, etc), it tells us little about how they interact with each other.

As Gav well knows, audience 2.0 isn't really an audience. An audience sits in the dark, only joined to each other by what they observe. This audience is noisy, they jump out of their seats. They interrupt the play. The critical thing about social media is that it's, well, social. What you have is more like a football terrace or a dance hall than a theatre/cinema/TV audience.

In SNA, you have two sets of measures:

1. Attributes of nodes - i.e. characteristics of entities or demographics if we're talking about people (which we may not be).
2. Attributes of links between nodes. Now a link between nodes is simply an interaction between them. We might have edited the same wiki page, we might communication in some fashion once a week about tennis, we might be having sex every night.

As a network analyst, you are often concerned with the interplay between node & link data - e.g. do 20-something males talk to more people more frequently on the topic of, say, organisational change. Or aftershave.

Link data is normally collected in 2 ways:

• Surveys of network participants - with all the problems of surveys (e.g. deliberate lying, wishful thinking, inaccurate recall).
• Automated collection of data (e.g. email usage patterns). In absolute terms, these are more reliable but give you no access to subjective evaluations of interactions - e.g. "that email was useful for me".

If we are going to understand the social media environment we need a solid understanding of 2. as well 1. There are several impediments to this:

• Collecting link data has traditionally been a lot harder than demographic data.
• Link data that is easy to collect is often difficult to interpret ("was that email exchange positive or negative?").
• Network-based metrics are poorly understood by marketers (in fact, by everyone).
• SNAs have tended to break down when you get more than 200+ participants - which in the consumer space is piddling. There are ways round this however.

Understanding social media and herd behaviour requires us to revolutionize our measurement techniques. Are we ready for that?