Big Data Rush- Data Is King

Big data is everywhere. It's created on farms, factories and phone lines. It's gathered in homes, hospitals and warehouses. You're even contributing to data pools by reading this article on your mobile phone, tablet or computer.


Tech geeks have long discussed the potential of capturing the data we create to drive new efficiencies, target services more directly and provide the complex calculations that can underpin exciting new technologies.


According to author Rick Smolan, big data is opening up "a completely new dimension" as information from mobile phones, smart devices, internet browsers, ATM cards, medical equipment, desktop devices in the workplace and much more is gathered and analyzed. This is affecting all manner of industries including, health, transportation and entertainment and is "like watching the planet develop a nervous system," Smolan says.

Yet the sheer scale and complexity of these developments can often be lost in a never-ending well of digits and decimal places.


Did you know, for example, that the amount of data captured globally is estimated to reach 40 zettabytes* (more on which later) by 2020? Sure, that sounds like a big number, but can you visualize what it actually means.

Not convinced about the rise of big data? This stat ought to change your mind.


According to research cited in the World Economic Forum's Global Information Technology Report 2014, 90% of the world's data was generated in the last two years. The remaining 10% was produced throughout the rest of human history.


Obviously, advancing technologies like the rise of mobile phone apps and social media platforms have a large part to play in this sudden explosion of data but the falling cost of storing data in recent years has also had a profound impact.

Unsurprisingly, the most developed regions of the world have raced ahead with the amount of information stored and collected.


According to the data in this graph, collated in the Big Data: Next Big Thing report from global IT trade organizations Nasscom, CRISIL and GR&A, the U.S. collected and stored more than 3,000 petabytes of data in 2012.


To put that figure into context, one petabyte contains enough space to store the DNA of the entire population of the U.S. and then clone them twice, according to analysis from Computer Weekly.


Meanwhile, large internet firms like Google or Apple store as much as 100 petabytes of data on their own servers, according to theTech Terms glossary.


As the report figures were captured two years ago, they are likely to have multiplied in the period since.


By 2020, the amount of data captured globally is expected to reach 40 zettabytes (there are a million petabytes in a zettabyte. A single zettabyte contains enough space to store 36 million hours of HD video), according to Nasscom.

The market for big data products -- defined as revenue derived from sales of related hardware, software and services to transmit and capture data -- totaled $18.6 billion in 2013, according to the Wikibon Big Data Market and Revenue Forecast 2013 - 2017.


That's more than the annual GDP of some small countries.


It also represents a growth rate of 58% over the previous year, emphasizing the growing focus and capacity of companies, governments and institutions to store the data we create.


And as this graph shows, the industry of data collection and storage that has sprung up in recent years has plenty of room for growth as more of our data is captured.

Smartphones have made us all contributors to the big data economy. Every social media post, video, picture and website visit can tell businesses something about their customers.


This graph from the WEF's Global Information Technology Report 2014 which analyzed data from Cisco, shows the sharp growth of mobile data traffic looking ahead to 2015.


True, the method of calculation chosen may not be the most reader friendly, but think of it this way: If the level of data created in 2010 equaled 100 bytes, then that figure is set to multiply by a factor of almost 20 by 2015.


For large tech companies such as Google, Apple, Facebook and Twitter this boom will offer a plethora of consumer data that they can store, analyze and use to tailor services to customers and advertisers.


Africa alone is expected to hit one billion mobile phone subscriptions by 2015, according to Informa Telecoms.


But alongside these opportunities, there are also issues surrounding privacy and security.


Concerns over how data has been shared or can be exploited by intelligence and security agencies like the NSA and MI5 has been an issue of major public debate in recent years, thanks in part to the disclosures of whistleblower Edward Snowden.

Where research leads, commercial adaptations tend to follow.


This graph from Research Trends Magazine displays the increasing number of peer-reviewed research papers exploring the topic of big data.


Experts working in the field suggest that big data will help power exciting new systems like the internet of things, where any physical item -- even everyday items like a chair or fridge -- can connect to the internet and make decisions based on their owner's data patterns.


But for these futuristic visions to happen as planned, there has to be a focus on training people with the requisite skills to make them a reality.


The 2012 Big Data: Next Big Thing report suggests that the U.S. alone will suffer from a shortage of 190,000 data scientists by 2018 if steps are not taken to encourage more people to learn the skills required to keep up with big data's booming growth.

Previous Post
First Post
Next Post
Automated Data Capture Methods and Benefits