It's amazing how many of these catch words really do make the rounds. The last few years have seen "The Cloud" bandied about as a must have for enterprise and it soon enough reached the consumer though the likes of iCloud, SkyDrive, DropBox and several other online solutions to old issues - store what you like on the internet.


"Big Data" is a term born from the cloud. It's not that challenges with large amounts of data is particularly new at all; it's that people are starting to proposition new ideas on how to solve some very difficult problems (old and new) which for the most part have always needed hard to access scale and processing power to solve. As you're no doubt already aware, The Cloud is much more than just about storage though and as soon as engineers start to join the dots on the benefit of scale, speed and redundancy, some truly innovative solutions start to appear. Solutions that have real world impacts in terms of costs, efficiency and reliability of digital services to name a few.

When you first hear the term "Big Data", perhaps the thought of massive data sets is the first thing to pop into one's mind. In truth Big Data is much more than that, and more than just understanding what to do with vast amounts of data. It is understanding that the "Big" portion can refer to analytics of disparate data sources, that a lot of data is not always a good thing (curation of Big Data) and that there needs to be a problem or challenge before a solution can be designed using Big Data propositions.

What is Big Data in a technical sense?
It's definitely a term of relevance. To some, 100 Gb may be Big Data. To others, it's Petabytes or even Exabytes. It's generally Big Data when the systems you have available in what you would consider to be normal operating scenarios can't cope with size of the data you have captured or sourced and further more your systems can't cope with the possible solution(s) required.

It's also about time. If you have a challenge that requires a solution to provide a result in a reasonable period of time, generally speaking only scale can allow you to create the solution. The you have a Big Data challenge on your hands.

At its core though, Big Data is really about mess. In a technical reference, it often is more about getting data into a useable format, into a questionable state and then how to query that result efficiently to gain insight and return a result for a further decision to be taken.

Take a large online/offline retail scenario. Imagine if you will, 3000 physical POS (point of sale) units. Now imagine how much data is flowing through each one of those units at any given time. Combine that data with online transactional data, mix in the back end data around stock and inventory control and supplier pricing. Now just to add some salt with the pepper, add multi-regional information from several countries and an ever present exchange rate.

Ok, you now have quite a lot of data which you have had to put into a very large storage scenario (thank you Cloud). But what you also have is a mess of data in different formats, different data representing many different products and items, different data that means certain things for one person and nothing to others. Data that may be updating and refreshing rapidly. If you were controlling that situation, aside from pulling your hair out, you may also have some challenges to solve just to get to the query stage, but in a real world sense, you have now got yourself - Big Data.

What needs to happen next, now that you have this Big Data? You need to really step back and consider the possibilities of the data to understand it, to mould it, to talk to it.

Sometimes, Data dies.
Some of the Big Data solutions I have seen over the last few years have missed the true value of the data. Yes, sometimes data dies. Data has a relevance expiry period, and to be honest, if you don't need to use data beyond a period in time - delete it (from your copied data set of course).
Just because your Cloud solution gives you 1000 cores at your command to process a query, doesn't mean you should be lazy about your data. Don't stress fat data - like the humble human, it can cause death. In this case, death of a query, death of a timeline, death of a job being completed properly. Every single piece of data included in a query set, has the ability to change results, skew perceptions and add complications and possible errors into the challenge.

What can you do with this data?
That is your challenge in truth. What value can you extract from your data? Amazon uses it to find out recommendation patterns, Google uses it to query a search request across the index of the internet, others use it to determine sales opportunities that are highly targeted with geographic insight into customer profiles. You can find insight and understanding within your organisation in some pretty meaningful ways once you start to consider the opportunities that the data you collect might have.

Splendid and Big Data.
Splendid has been engaged with Microsoft for years, using Azure for more than just scaled cloud solutions. Azure is a Big Data solution and effectively allows you to throw "cores" or processors at the problems that Big Data represents. You can process very rapidly if you are willing to pay for those processors. Google too offers a Big Data solution on their cloud - indeed both allow Hadoop instances and both allow all this to be done on Linux based VM's that you can move between the services at will. There are many other providers in the marketplace as well as tooling that can help work with and succeed at these scales.

Splendid has also used various non-relational database solutions in the past and present - critical to enabling Big Data queries without the size and performance constraints or traditional databases such as SQL Server. We have worked with MongoDB on various social applications which really couldn't return live views of trends, without being able to query data that builds in size and complexity at a geometric rate.

The secret to our successes and those of our clients working with Big Data is always with the quality of Data Scientists that we use. These guys are the ones that create the algorithms, the ones that understand fractured data and help us deliver those results.

If you believe you have a Big Data challenge in front of you, Splendid can help deliver meaningful results that continually change and adapt to your growth into the future.

The above headline would once have caused serious scratching of heads for many or, put a little more highfalutinly, generated a spasm of cognitive dissonance throughout the creative industries. No more however. Whether we've quite yet grasped the fundamental impact that Big Data delivers, it's already here and it's everywhere. It's affecting the way we perceive the world, the manner in which we're engaged by brands and the way businesses must now organise themselves. It isn't a trend that will fade away in the not too distant future. It's here to stay and the only option for the creatively minded is to fully embrace it.

Last year I was fortunate enough to be working at the BBC to help deliver their (frankly fantastic) Olympics coverage. Specifically, I was leading the design teams developing the online and connected-TV video products. Every minute of the action was made available through them, both live and as catch-up, with every frame augmented by split-second performance data on every athlete. During the 17 days of the Games, 2,500 hours of video and supporting statistics for the 10,000 athletes were delivered via 27 concurrent live streams, resulting in 2.8 petabytes of data outputted nationally every day. Those are absurd figures to design against, and it took numerous tough decisions about what data should be available to the viewing audience and how best to design the end interfaces. I've never been more reminded of the design-truth that the greater the complexity of information, the more elegant a delivery method is demanded.

Above and beyond creatively delivering Big Data, there's the onward march of Big Data defining the things we create. Market insight has always shaped product development on some level, however, the granularity of insight Big Data affords is very different. Companies born of the online revolution naturally have an innate understanding of digital insight and proposition development, and right now Netflix is leading the pack. Known for smart content recommendations, it's now shifted into the smart content production business. Its first step is a remake of the classic 80's BBC drama, House of Cards, this time starring Kevin Spacey in the lead role. Netflix's decision wasn't based upon a love for off-beat British television however - Big Data about their users consumption of content steered the entire production, digitally matchmaking viewers of the original series, films starring Kevin Spacey, and those directed by David Fincher, who oversaw the remake. This article by Andrew Leonard raises an interesting question however - if content makers can back a sure fire hit every time through granular insight of past customer interest, where do the surprises come from? Solely developing creative endeavours upon such a methodology may sleepwalk us into a world soundtracked by Magic FM classic hits, all day long and for the rest of our lives.

Fortunately I suspect that's not going to happen, for whatever you make of Netflix's creative decision, the process used is indeed innovative. And innovation will always keep us on our toes. The challenging question that Big Data poses is how can businesses capitalise on the opportunity being presented. Nesta's recent report, 'Rise of the datavores' provides some fascinating insight into the characteristics of British businesses that are grabbing and running with the idea. The most pertinent aspect of the report is the very clear recognition that irrespective of the amount of customer data a company has access to, if it doesn't have the analytical capabilities to infer any meaning, it's all just noise. Even when insights are garnered, if the organisational structure and business culture doesn't afford rapid action, the insights will be lost and the business continually positioned behind the curve.

Big Data is hitting every aspect of society, and at a really crazy rate. As providers and creators of digital products and services, it's time we get our heads around it. Whether that's figuring out how we express all that information in a comprehendible and actionable manner for our customers, or deciding how best to navigate our working lives through the deluge of it, it's here and the opportunities need to be grasped.

We have been working with BBC Worldwide and Microsoft Advertising to launch a new version of the Top Gear Windows 8 app featuring custom ad formats for the new Jack Reacher movie. This interactive advert was produced by Microsoft Advertising and features touch gestures to navigate around the content using the gear stick, cars feature pretty heavily in the film.

It's nice to see James Corden likes our Windows Phone app for The Times. You can view the full advert on YouTube, part of the Reinvented Around You campaign.

Splendid and BBC Worldwide have been working hard on the new Windows 8 app for Top Gear News and are happy to say its now live in the store. I think its best summed up in best Tog Gear fashion here "Have you always wanted more Top Gear in your life? Then good news! Here is our brand new Top Gear News App, with all the latest news, videos and random nonsense direct from the Top Gear team."

The Windows 8 app was designed and built by the Splendid team in close collaboration with
BBC Worldwide and Microsoft, and it featured on as a Spotlight app on the Windows Store when
it went live.

Key features;
All the latest news, rants, video and reviews live from Top Gear HQ
Behind the scenes exclusives from the three presenters
Your favourite clips from Top Gear TV

A full case study will follow, but in the meantime you can follow the development process in a series of articles on ubelly.

Splendid attended the Imagine 2012 event hosted by Microsoft Advertising, to witness the launch of Windows 8 to the advertising industry. There were some great speakers and it proved to be an excellent evening.

Highlights for us were: Anna Kirah: Senior Socio-anthropologist, Toby Barnes: AKQA Product Strategy Director and Stephen Kim: GM, Yarn, a Microsoft Advertising Creative Lab. The event was topped off with a "fireside chat" with Steve Balmer. Splendid also made an appearance with sneak preview of our work with BBC Worldwide and a few vox pops on Windows 8, presented by director Paul Bishop.

The hype has now turned to reality. So whats it like?

It's no secret we have been working on a number of Windows 8 apps for our clients so you may think we are biased, but I thinks it's fair to say we are really impressed with Windows 8 and the hardware that goes with it, be that the Surface Slate or the new Nokia Phones.

We have had new Microsoft Surface Slate for a few days now and for the first time Microsoft have a complete hardware/software product on the market, by that we mean software and hardware packaged into a covetable, desirable device. And it really is quite striking!

For once a Windows device that doesn't draw sniggers of derision from our design team. Instead it's been the subject of a lot of curiosity, around the hardware as much as the new user interface.

Microsoft have bravely stepped away from the WIMP (Windows, Icons, Menus, Pointers) UI we are all familiar with, and lead with a touch-first solution. In fact, the term 'Windows' may be a misnomer in this case, as the OS is an entirely new chrome-free, pane-based UI. Various manufacturers have already released touch solutions of course, but none applied this to desktop too. It'll be interesting to see how sales of touch monitors go as a result.

Back to the more familiar realm of the tablet, and there is a learning curve with Windows 8, not a huge one. Once you are past this it becomes intuitive and easy to use. It really is something you could use on a day to day basis and the general opinion round the office (mainly Mac users if I am honest) is that "it's cool". Not the most in depth review but really its probably what will count in terms of adoption and sales. Big thumbs up from us, well done Microsoft.

Splendid were at the launch of the Modern Jago last night. Microsofts new creative space in Shoreditch. It's a great space in the Rochelle School on Arnold Circus, we plan on using it for workshops and talking about the things that matter to us, great user experience and cutting edge technology.