Big Data: A Force for Good...or Evil?
We hear a lot about “Big Data” today. The idea of capturing enormous bodies of data and then analyzing that data is heralded for its potential to understand the cosmos, cure disease, provide safety for our citizens and save the environment… For all its promise, though, there are as many voices that decry it as the very end of privacy, liberty and security. What we have are two enormous and very polarizing viewpoints about Big Data in modern society.
As with all “big” polarizing issues, there is a huge middle ground; places where the good outweighs the bad, where we all can agree that benefits can be found. In this space, applied data analytics provides unprecedented value with virtually no downside.
At its heart, data analytics is really about taking a closer, more granular look at things we are already looking at, in the attempt to understand them better, not just observe them better. As I write this, a scrap of the lyrics from an old Eagles song comes to mind:
”The more I know, the less I understand. All the things I thought I knew, I’m learning again.”
This fits data analytics today. While we have been collecting data ever since the advent of writing, we have only recently begun to analyze what we collect. We did this first in the 20th century with human analysts—a good example being the brilliant Alan Turing and his Benchley Park staff breaking the Enigma code, working with a very constrained set of data to find deep associations. Others, like marketing analysts, find broad correlations in consumer preference.
Today, with our digital infrastructure, an explosion in sensor technology (it seems we humans have a real penchant for collecting data) and network communications means we have far more data than we can ever hope to analyze. (It has been said that, for every hour of data taken by a surveillance drone, 40 man-hours of analysis is required to glean useful information). This “information lag” prevents anything close to “real time” reaction to new information. As such, automated data analytics is imperative.
So “Big Data” uses computers to “learn” and look for associations that human analysts could not possibly find, and today, they analyze medical records, financial records, public utilities like water, power and gas. The airplanes we fly in most often have real-time data collection on engine operation; oil refineries and pipelines likewise are monitored continuously. The list goes on and, in each case, the goal is more effective treatment, faster corrective action, prevention of accidents and greater security.
But this is the “Connected Battlefield” blog; you might well ask what this has to do with the military? It’s a good question. The answer is: “Everything.”
Intelligence has always been a part of military strategy. Consider Sun Tzu’s The Art of War, written in the 6th century BC:
“If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.”
It’s about knowing
Note the use of the word “know”—not see, or measure, but know. Today, with satellites, drones, airborne radars, ground-based sensors, and many other data collection devices, we can know ourselves and our enemies—but only if we can quickly and effectively analyze what we measure and produce meaningful, timely information. This protects our forces; keeping them healthy and secure and improving their efficiency as well.
This is our job. We provide the high-powered computers that quickly and reliably process data, and we provide analytic software that spots trends and suggests actions. Every minute of every day, our hardware and software are at this valuable work.
“Big Data” is here. It was not created in the 21st century as some portent of the apocalypse. It has evolved—and with it, the tools to make it work for the greater good have evolved. Like all powerful tools, it is not what they are that works for good or ill; rather, it is how we use them that matters.