(Stephen Parker, Head of Cloud Strategy, NewLease)
After “Cloud”, “Big Data” is one of the buzz words at the moment. I was recently asked how they were connected and for some examples. Following is a summary of my reply:
One of the biggest challenges for all businesses will be that Big Data will require staff who can make Big Judgements.
- Analysis resulting in “new” results will require a skill set that is not common in many businesses.
- The wrong judgments will be potentially more damaging than no judgment.
- There is an interesting article in the Harvard Business Review about this (http://hbr.org/2012/04/good-data-wont-guarantee-good-decisions/ar/1)
Some of the obvious examples of Cloud and Big Data are with organisations that have large existing data sets. The connection is that all of these benefit from the cloud attributes of low cost storage, on-demand/elastic compute and consumption based billing:
- Large retail businesses such as Supermarkets have huge loyalty card and buying data. By mapping this to other datasets such as demographics, weather patterns etc they can enhance stock management and deliver highly targeted advertising. E.g. Target’s Big Data spend analytics can predict early stage pregnancy and send related marketing (http://www.dailymail.co.uk/news/article-2102859/How-Target-knows-shoppers-pregnant--figured-teen-father-did.html)
- Pharmaceuticals can run more potential drugs through early stage simulation and modeling hence increasing the possibility of finding the next “big” one
- Governments in general. Cross functional and departmental analysis has the potential to reduce government wastage and enhance services to the tax payer (you and me!!). However this is a highly sensitive subject with all sorts of privacy implications.
- Health services. Again ability to centralize health data with external data sets such as demographics has the ability to predict viral outbreaks and implement timely prevention.
An interesting future example of real public cloud and big data will be the Square Kilometre Array (http://www.skatelescope.org/). This will be deployed in WA (along with South Africa). Just as the need for simplified and open access to information at CERN drove Tim Berners-Lee to lay the foundations for the Internet, so the SKA has the potential to drive a similar transformation with the Cloud.
- It is predicted that the SKA when fully live in 2024 will generate 1TB/sec of data
- This is a “public” project supported by multiple nation states who will all want research access to the data.
- The volumes and compute are ideally suited to the cloud model (http://www.skatelescope.org/uploaded/8762_134_Memo_Newman.pdf)
Big data is often seen as a preserve of big organizations. Cloud consumption based pricing for compute will democratize access to Big Data for Small & Medium Businesses.
- Large data sets will be made available by various organisations, both public and private sector.
- SME’s will have some arrangement to access the data. This maybe free or on a commercial basis.
- They will then use elastic compute resources with consumption based pricing to analyse these data sets
- Service providers will provide the aggregation points, the analysis tools and billing