BigData Training- @ Chennai - BigCHALLENGE

INTRODUCTION: THE CHALLENGE OF BIG DATA

The amount of data in the world is being collected and stored at unprecedented rates. A study by IDC Gantz & Reinsel, (2011) indicates that the world’s information is doubling every two years. 

Also the IDC study by Gantz & Reinsel (2011), mentions that the world created a staggering 1.8 zettabytes of information (a zettabyte is 1000 exabytes), and projections suggest that by 2020, we’ll generate will generate 50 times that amount.

Big Data has been defined as, when data sets get so large, that traditional technologies, techniques, and tools for extracting insights are no longer useful in a reasonable timeframe and cost-effective manner. This has spawned a new generation of technologies and corresponding considerations. Desai, Kommu & Rapp (2011) examine the cause of this explosion of Big Data, the following factors dominate:

·         Mobility trends: Mobile devices and sensor proliferation;

·         New data access: Internet, interconnected systems, and social networking;

·         Open source model: Major changes in the information processing model and the availability of an open source framework.

What distinguishes Big Data from data in the past, however, is not just its vast volume. The defining features of Big Data are also its variety—the sources and types of data being collected—and its velocity, the speed at which the data is flowing through the networked systems. Studies like Cisco Virtual Networking Index by Barnett, (2011) estimate that in 2016, global IP traffic will reach 1.3 zettabytes per year or 110.3 exabytes per month. Moreover, it is anticipated that there will be 19 billion networked devices by 2016.

One of the most interesting aspects about Big Data is that that unstructured data is the fastest growing type of data. Unstructured data refers to information that either does not have a predefined data model or does not fit well into relational database tables. 

Examples of unstructured data include imagery, sensor data, telemetry data, video, documents, log files, and email files. The challenge is not only to store and manage this vast mix, but to analyze and extract meaningful value from it—and to do so in a reasonable timeframe and at a reasonable cost.