Atlantic.Net Blog

The Challenge and Promise of Big Data

Editorial Team
by Atlantic.Net (219posts) under HIPAA Compliant Database Hosting

As the Internet has grown, so has the amount of data and the extent to which data is valued – most notably by the government and publicly traded companies, although even small businesses appreciate the insights they collect from Google Analytics and similar services. Stockpiling and scrutinizing data is considered an extraordinarily powerful way to deliver better products; understand how people use websites and applications; track employee performance; determine the most effective marketing language; and more.

According to projections released by International Data Corporation, the big data market will maintain a CAGR of 26% through 2018, when sales of big data services will hit $42 billion.

The value of the market tells us that businesses are investing large sums of money in companies that specialize in the collection, comprehension, and integration of vast pools of data – in other words, services that can turn the information into better results.

How big of a focus is big data? Well, the spaceship people are in the game.

Chris Mattman of NASA says of his employer’s research in this area, “NASA in total is probably managing several hundred petabytes, approaching an exabyte.”

Forbes contributor Joshua Steimle notes that an exabyte is an extraordinarily vast amount of information, 1 billion gigabytes.

Awash in a Sea of Numbers?

With the technology we now have available, data can accrue so rapidly for enterprises that it amounts to overload for traditional databases – both in terms of keeping it and making any sense out of it. Companies don’t have to build rockets in order to feel somewhat overwhelmed by the sheer enormity of the data at their fingertips, argues Steimle.

Chris Riquier, the Asia-Pacific head of Taylor Nelson Sofres, says he has seen the impact in another area: “Market research was founded on surveying and polling.” Now, instead, it is done in real time with information flowing in from social media, search engines, and numerous other sources. Thanks to the focus of technologists on data science and the continual refinement of analytics, Riquier believes that our “ability to react to the market and make decisions has changed drastically.”

Finding meaningful patterns within big data has been a mystery to many firms. However, data engineers are building systems so strong that the trend toward amassing and processing data as quickly as possible is all but inevitable, per a 2014 Harvard Magazine report. The article suggested that innovative “big algorithm” formulas will allow firms to benefit from connecting and integrating data stores, as well as converting the numbers into more robust and user-friendly imaging.

Drawing on long-established research techniques from the fields of physics and astronomy, technologists and cultural experts bring their different perspectives to bear on the topic for mutual benefit.

Big data authors Viktor Mayer-Schonberger and Kenneth Cukier note that data analytics isn’t just about individual choices but about completely altering the way that businesses make big-picture decisions – after all, predictive analytics is intended to live up to its name, giving those who use it a logically based, all-seeing crystal ball.

Big data successes include:

Google – Google hypothesized that certain search terms were probably indicators that people were beginning to have flulike symptoms. By applying their guesses to real-time searches, the company was able to figure out where the flu was spreading faster than the CDC could.

Canadian Bank – Across the border, Canadian Bank is “using Hadoop, an open source software framework created by Apache,” Steimle explains, “to roll out a program that enables the identification of money laundering and fraud.”

Data & everyday life

NASA and Google are high-profile instances of the focus on big data, but its potential is much broader, extending to any business wanting to get attention online. It is the fuel that powers search engine optimization and social media optimization. Entrepreneurs can use Google’s Webmaster Tools, combined with analytic data from social software, to develop a more successful online presence.

Joe Hall, who helps clients improve their Internet presence through his company Hall Analysis, comments that there are two different basic ways data can be analyzed to improve search engine results:

  1. Exploring the patterns within the voluminous data that is directly related to a particular business.

One of Hall’s clients had more than 15 million backlinks – meaning that many, many webpages were sending their traffic to his client.

At that level it changes the rules for backlink analysis,” Hall comments. Determining the patterns within the data become significantly more complex.

  1. Garnering a more sophisticated understanding of their business’s current position and the dynamics in the market as a whole.

Looking at the relationship between different variables through analytics software allows businesses to better understand how to rank prominently and how to get more people to respond to a listing when it does show up in the search engine. Two relevant fields of study are correlation analysis and user behavior analysis, says Hall.

Revving your big data engines

One technology that has been fundamental to data scientists and companies utilizing their services is premium cloud computing. After all, turn-key cloud hosting solutions are based on optimizing speed, reliability, and efficiency through virtualized distribution of resources.

Computer scientist Geoffrey Fox of Indiana University has even noted that the cloud is often faster than a supercomputer.

By Kent Roberts

Cloud hosting is just one of four services we offer – we also offer managed, dedicated and HIPAA compliant database hosting services.

Start Your HIPAA Project with a Free Fully Audited HIPAA Platform Trial!

HIPAA Compliant Compute & Storage, Encrypted VPN, Security Firewall, BAA, Offsite Backups, Disaster Recovery, & More!

Start My Free Trial

Looking for HIPAA Compliant Hosting?

We Can Help with a Free Assessment.

  • IT Architecture Design, Security, & Guidance.
  • Flexible Private, Public, & Hybrid Hosting.
  • 24x7x365 Security, Support, & Monitoring.
Contact Us Now!
Stevie Gold Award Med Tech Award

SOC Audit HIPAA Audit HITECH Audit

Case Studies

White Papers


HIPAA Partners

Recent Posts

Get started with 12 months of free cloud VPS hosting

Free Tier includes:
G2.1GB Cloud VPS Server Free to Use for One Year
50 GB of Block Storage Free to Use for One Year
50 GB of Snapshots Free to Use for One Year

New York, NY

100 Delawanna Ave, Suite 1

Clifton, NJ 07014

United States

San Francisco, CA

2820 Northwestern Pkwy,

Santa Clara, CA 95051

United States

Dallas, TX

2323 Bryan Street,

Dallas, Texas 75201

United States

Ashburn, VA

1807 Michael Faraday Ct,

Reston, VA 20190

United States

Orlando, FL

440 W Kennedy Blvd, Suite 3

Orlando, FL 32810

United States

Toronto, Canada

20 Pullman Ct, Scarborough,

Ontario M1X 1E4


London, UK

14 Liverpool Road, Slough,

Berkshire SL1 4QZ

United Kingdom