The Real Reason Systems Fail

It seems as if every week there’s another news article bemoaning the state of data integration within some large enterprise. Mission objectives are stymied because “systems don’t talk to each other”. Intelligence failures are due to “incompatible data”. The surprise and outrage expressed would make the lay reader think that this is a recent trend, but they’d be wrong in this notion. The “Data Integration Problem” has been around since the first humans began to speak. Most practitioners and experts who work in the software version of the problem space haven’t realized this, but it’s true.

What is “data” in the modern sense? Most people think that it is “information”, the detailed “facts” of a modern culture. This colloquial understanding is a major simplification, one which is at the root of the Data Integration Problem. It is the reason why most people, even seasoned experts, are constantly surprised and frustrated when the monster appears, seemingly out of nowhere, before them.

So what is data?
Data is CODE.
Data is REPRESENTATION.
Data is SYMBOLOGY.

What this implies is that without someone who can decode it – an INTERPRETER – data is nothing. Let that sink in for a minute.

Data is nothing without INTERPRETATION.

What does this mean? Well for one thing it means that without an interpretation, there is no way to even recognize that data exists. And without an interpreter, there can be no interpretation.

So when we think about all of the data being generated and passed around in our modern world, the question arises: Who is the interpreter that gives data its meaning? Well obviously it’s us. The computer doesn’t understand the data it contains! No matter how we might try to anthropomorphize them, computers are still just as dumb as the lumps of metal, plastic and sand from which they are constructed. The systems that we humans create using these computers are just that – mechanical systems which manipulate physical media, morphing symbols from one representation to another. Everything a computer does is devoid of intrinsic meaning until some human comes along and interprets the symbols.

Imagine computer systems after apocalypse. Imagine the systems of the stock exchange, or the weather service, or any of the thousands of other automated systems that may run unattended by their now defunct human inventors. Now answer the question: without humans to interact with them, do they produce anything? Is there any content to them without, ultimately, some human being to interpret their output?

This is more than that old saw about a tree falling in the forest. Consider some famous examples of symbols that have lost their meaning:

  1. Cave paintings of Lascaux depict hunts and animals, but what did our pre-historic cousins intend when they outlined their hands on the walls?
  2. Until the Rosetta Stone was found, Egyptian hieroglyphics had lost their meaning in the world.
  3. When the Confederacy fell, Confederate currency lost its meaning and value.

The Data Integration Problem, simply stated, is caused by the fact that data is symbolic code onto which some group of humans has projected meaning. Meaning, therefore, is local to the humans doing the projection. Without the knowledge of how meaning was projected onto the symbols, the information they contain cannot be retrieved in any complete sense.

Only the people who have projected their meaning onto the symbols (or in special instances who share significant experiences of the world with those people who have) are able to interpret the data correctly. In any large, complex, enterprise, where business necessitates that small groups complete their own missions expeditiously and with vigor, who should be surprised that locally defined data doesn’t integrate well from one end of the enterprise to the other?

Really, the answer to this question should be “nobody”.

This has been true since humans (and possibly our predecessors) first started making symbols.

%d bloggers like this: