Small Data in Big Orgs: Not Sexy Enough

Working in a large company is frustrating. One way in particular is when trying to quickly *do* something with interesting pieces of small data. When enterprise social networks are used well, employees share all kinds of wonderful insights that benefit the organization. Much of what is shared is individual insight or ideas, based on real front-line experience. The research scale is n=1, but it doesn't mean it is worthless. Quite the contrary. 

Here's an example (and what inspired this post). Just this week I saw a novel product placement idea from a field sales rep on our ESN. They had negotiated something different with a local store. I was impressed and asked them how they would measure the impact of the idea. They came back a few days later with actual figures that showed it was a good idea. But that's where the story ends. 

I asked around (globally) if we could take the successful idea and run it as a prototype in a few more stores. Could we use the same measures of success for a short, defined period - basically run a very defined sprint. Then, should the idea work again in other stores, scale it as fast as possible. In my mind, why couldn't you go from a small, tested, successful prototype idea to a bigger rollout in a few weeks? The opportunity window might be small, but the effort would likewise be small. The result may not scale, but would open up a new level of engagement with the sales team.

As I've discovered many times, systems and processes are not set up for small data experiments. To run a small team, agile experiment, you need marketing, sales, product teams and others to do two things:

  1. be represented on the experiment team by someone who has been devolved decision-making authority; or
  2. be OK with not being part of the experiement team

Instead, all want to be involved and have significant input. Or, in this case, the opportunity was too small to even register an interest.

It's a shame. Big organizations are getting better at looking at their big data, but they are missing out on the small. Big data is sexy. Small data isn't. Big data is where the big rewards are (so say the marketing consultants). But as Rufus Pollock of the Open Data Foundation says

"Size in itself doesn't matter – what matters is having the data, of whatever size, that helps us solve a problem or address the question we have."

The field sales rep I worked with had solved an interesting problem in an innovative way. Unfortunately, we couldn't go any further.

I've always been a connector. People to people. People to information. I see many small data opportunities, just waiting someone to make the right connections. But it's shame there is little operational support for it in big organizations. As I've written before, the future of organizations depend on how they leverage networks: for innovation and continued longevity. At the moment, the lack of understanding of the benefits of small data is another reason that big legacy organizations cannot harness the innovation network potential they possess. 

I'll keep plugging away. The flip side of being a connector in a big organization I find heartwarming, rich seams of ideas and collaboration every day (even if no-one else notices).