Can mid-market companies really use Big Data and Predictive Analytics?

The question of the day is whether mid-market companies can really use and gain benefit from Big Data and Predictive Analytics. Although many companies in this category don't have a great deal of expertise in this area, with a bit of help from their friends, the answer can be a resounding "Yes."

Before I can talk about how mid-market companies can benefit from using this technology, I'll take a moment to define the terms.

Big Data

"Big Data" is a concept that is being flogged by just about every supplier of IT solutions today. It, in simple terms, is tools, processes and procedures allowing the creation, manipulation and management of very large, rapidly changing data sets that include both unstructured (documents, spreadsheets, presentaitons and the like) and structured (databases and operational data coming from IT systems and software) data.

"Big Data" is a catchphrase for a mainstay of the world of high performance and technical computing. It is a technology that has been used in the financial services, healthcare, research and engineering worlds for quite some time.

Predictive analytics

Predictive Analytics is the use of artificial intelligence, modeling, statistics, and pattern detection algorithms to sift through the mounds of data to identify behavior patterns that could foretell a negative outcome.

When applied to IT operations data, it processes the log and performance data created by operational systems in order to learn how systems work together and identify the anomalies that appear before workload slowdowns, outages or security intrusions.

When applied to customer facing systems, it can process data on web traffic, customer orders, shopping carts, manufacturing systems and the like to gain a deeper understanding of customer requirements.

As one would expect, developing this sort of tool requires exceptional expertise in numerical tools that can analyze data coming from a variety of sources and in a multitude of formats to determine what is "normal" behavior and what is an anomaly.

Over the years, the technology supporting both "Big Data" and "Predictive Analytics" has been refined and is ready to be deployed in helping companies address the complex, multidiscipline, multi-system, multi-tier operational systems they use every day.

Why should mid-market companies care about this?

Wouldn't it be nice if it would be possible to discover:

  • Previously hidden customer preferences, such as what is the best day of the week to sell blueberry donuts
  • Improve productivity, such as what data and applications does staff use the most and what applications are seldom used.
  • Reduce costs, such as is your company paying to use services and never using them?
  • Increase revenues, such as would changing product packaging or messaging increase sales?

Big Data combined with predictive analytics can help mid-market companies find answers such as these.

This is where suppliers, such as IBM, HP and others come in. They are highly distributed companies. They have functions spread all over the globe.  Engineering, sales, marketing and support staff are distributed as well. They have the tools and the knowhow to make this work.  Small companies can certainly rely on these partners for help defining and executing a reasonable remote worker program.

Challenges of Big Data and Predictive Analytics

Using Big Data and Predictive Analytics can be challenging for large companies. Can mid-market companies ever hope to harness this technology for themselves?

  • Can these technologies really deliver useful answers? — the answer is yes, but, it is important to know what you're looking for and have good data. Remember the "GIGO" rule of IT — Garbage in, Garbage Out. Bad data or bad analysis will result in bad conclustions.
  • Do I have the expertise to use these technologies? — Even large companies may not have all of the expertise needed to use these tools.  Luckily, mid-market companies can get extensive help from partners like IBM, HP, Oracle, Dell and the like.
  • Do I have to acquire systems and software to put these technologies to work? — The answer is no. Suppliers such as IBM, Amazon, Cloudera, and Microsoft are offering "Hadoop-as-a-Service" to companies who don't want the burden of creating their own Big Data computing environments.

Mid-market companies now can have access to the same tools that the biggest companies are using to better understand their customers, optimize their product and service offerings, and increase their revenues. It isn't necessary for mid-market companies to take on Big Data and Predictive Analytics by themselves. They don't even need to acquire new systems or software to take advantage of these tools.  They do need to know what questions they need answered and have good data to evaluate.


This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I've been compensated to contribute to this program, but the opinions expressed in this post are my own and don't necessarily represent IBM's positions, strategies or opinions.

Kusnetzky Group LLC © 2006-2014