You Can’t Store Meaning In Software

I’ve had some recent conversations at work which made me realize I needed to make some of the implications of my other posts more obvious and explicit. In this case, while I posted awhile ago about How Meaning Attaches to Data Structures I never really carried the conversation forward.

Here is the basic, fundamental mistake that we software developers make (and others) in talking about our own software. Namely, we start thinking that the data structure and programs actually and directly hold the meaning we intend. That if we do things right, that our data structures, be they tables with rows and columns or POJOs (Plain Old Java Objects) in a Domain layer, just naturally and explicitly contain the meaning.

The problem is, that whatever symbols we make in the computer, the computer can only hold structure. Our programs are only manipulating addresses in memory (or disk) and only comparing sequences of bits (themselves just voltages on wires). Now through the programming process, we developers create extremely sophisticated manipulations of these bits, and we are constantly translating one sequence of bits into another in some regular, predictable way. This includes pushing our in-memory patterns onto storage media (and typically constructing a different pattern of bits), and pushing our in-memory patterns onto video screens in forms directly interpretable by trained human users (such as displaying ASCII numbers as characters in an alphabet forming words in a language which can be read).

This is all very powerful, and useful, but it works only because we humans have projected meaning onto the bit patterns and processes. We have written the code so that our bit symbol representing a “1” can be added to another bit symbol “1” and the program will produce a new bit symbol that we, by convention, will say represents a value of “2”.

The software doesn’t know what any of this means. We could have just as easily defined the meaning of the same signs and processing logic in some other way (perhaps, for instance, to indicate that we have received signals from two different origins, maybe to trigger other processing).

Why This Is Important

The comment was made to me that “if we can just get the conceptual model right, then the programming should be correct.”  I won’t go into the conversation more deeply, but it lead me to thinking how to explain why that was not the best idea.

Here is my first attempt.

No matter how good a conceptual model you create, how complete, how general, how accurate to a domain, there is no way to put it into the computer. The only convention we have as programmers when we want to project meaning into software is that we define physical signs and processes which manipulate them in a way consistent with the meaning we intend.

This is true whether we manifest our conceptual model in a data model, or an object model, or a Semantic Web ontology, or a rules framework, or a set of tabs on an Excel file, or an XML schema, or … The point is the computer can only store the sign portion of our symbols and never the concept so if you intend to create a conceptual model of a domain, and have it inform and/or direct the operation of your software, you are basically just writing more signs and processes.

Now if you want some flexibility, there are many frameworks you can use to create a symbollic “model” of a “conceptual model” and then you can tie your actual solution to this other layer of software. But in the most basic, reductionist sense, all you’ve done is write more software manipulating one set of signs in a manner that permits them to be interpreted as representing a second set of signs, which themselves only have meaning in the human interpretation.

Q&A: Meaning Symbol Sign and Mind (Part 2)

On one of my recent posts, a commentor named “psycho” asked me some very good questions. I decided I needed to respond in more detail than just a single comment reply. I respond in pieces below, so just for context, here is psycho’s entire original comment.

But if you take more meanings, and put them together to get yet another meaning. Don’t you feel like those meanings were again like symbols creating a new meaning?

In my understanding, every bit of information is a symbol – what is represented by the invididual neurons in the brain. And if you take all related bits (that is neurons, symbols), and look at it as a whole, what you get is meaning.

The sentence is a symbol, and it is made of word-symbols. And the list of word-symbols makes a meaning. Which, when given a name (or feeling), becomes a symbol, that can be further involved in other meanings.

I’ll respond to each paragraph in separate posts, in order to get all of my thoughts down in a reasonably readable fashion. Part one covered the first paragraph. Here is part two where I cover the rest of my thoughts.

Symbols in the Mind

In my understanding, every bit of information is a symbol – what is represented by the invididual neurons in the brain. And if you take all related bits (that is neurons, symbols), and look at it as a whole, what you get is meaning.

I’m not a neurologist or any kind of brain scientist by any means, so I could eventually be proven wrong on this, but what a neuron represents, to me is not a symbol and not a sign and not a specific meaning. I know I read somewhere of a brain experiment (using MRIs I think) where the image of Jennifer Aniston presented visually during a brain scan caused only a single neuron to fire. I recall that the interpretation given was that the entire concept of “Jennifer Aniston” was stored in one singular neuron.

I guess I just don’t buy it. What if the meaning of that neuron was more along the lines of “a famous person whose name I forget” or “I recognize a face I’ve seen on ‘Entertainment Tonight'”? The fact of it is, the experimenters drew a conclusion on a correlation that not even their subject would be able to explain or confirm.

Then there is some hypothesis that memories and meanings are distributed across the brain in such a pattern as to suggest more of a holographic storage mechanism (where damage in one area of the brain is overcome by stimulation and growth and retraining).

I think that memory and meaning is essentially an EXPERIENCED thing. That the physical stimuli produce a complex of sensations through re-activation of neurons that causes the brain itself to “sense” the memory. I don’t think this qualifies as a symbollic sensation, being a much more holistic, “analog” experience not unlike the original. If every bit of information were a symbol, then I think we’d be just as hard-wired as computers to recognize only one set of sensations and meanings. Our experience being more fluid, it allows us to be much more creative in the aspects and portions of sensation that we recognize and name. As an individual I have full freedom to separate the signal from the noise, the foreground from the background, as I fancy. I can “slice and dice” me experience of sensation in anyway that I find meaningful, and if I communicate it to you, then you can see what I see just like that. In other words, working with the “analog” of my sensations is a much more powerful, creative endeavor than merely encoding and decoding “digital” symbols.

That’s my two cents on that thought.

Q&A: Meaning Symbol Sign and Mind (Part 1)

On one of my recent posts, a commentor named “psycho” asked me some very good questions. I decided I needed to respond in more detail than just a single comment reply. I respond in pieces below, so just for context, here is psycho’s entire original comment.

But if you take more meanings, and put them together to get yet another meaning. Don’t you feel like those meanings were again like symbols creating a new meaning?

In my understanding, every bit of information is a symbol – what is represented by the invididual neurons in the brain. And if you take all related bits (that is neurons, symbols), and look at it as a whole, what you get is meaning.

The sentence is a symbol, and it is made of word-symbols. And the list of word-symbols makes a meaning. Which, when given a name (or feeling), becomes a symbol, that can be further involved in other meanings.

I’ll respond to each paragraph in a separate post, in order to get all of my thoughts down in a reasonably readable fashion. Here is part one.

Construction of Symbols

But if you take more meanings, and put them together to get yet another meaning. Don’t you feel like those meanings were again like symbols creating a new meaning?

I try to make a very strong statement of the difference between symbols, signs and their “meanings”. Perhaps I’m being too analytical, but it allows my to think about certain types of information events in a way I find useful in my profession as a data modeller. So let me try to summarize here the distinctions I make, then I’ll try to answer this question.

First, in my writings, I separate the thing represented by a symbol from the thing used as the representation. The thing represented I call the “concept” or “meaning”. The thing which is used to represent the concept I have termed “the sign”.  A symbol is the combination of the two. In fact, a specific symbol is a discrete object (or other physical manifestation) built for the express purpose of representing something else. That specific symbol has a specific meaning to someone who acts as the interpreter of that symbol.

As I have come to learn as I continue reading in this subject area, this is a somewhat ideosyncratic terminology compared to the formal terms that have grown out of semiology and linguistics. To that I say, “so be it!” as I would have  a lot of re-writing to do to make my notions conform. I think my notions are comparable, in any case, and don’t feel I need to be bogged down by the earlier vocabulary, if I can make myself clear. You can get a feel for some of my basic premises by poking around some of my permanent pages, such as the one on Syntactic Media and the Structure of Meaning.

There is obviously a lot of nuance to describing a specific symbol, and divining its specific meaning can be a difficult thing, as my recurring theme concerning “context” should indicate. However, within my descriptive scheme, whatever the meaning is, it is not a symbol. Can a symbol have several meanings? Certainly. But within a specific context at a specific time, a specific symbol will tend to have a single specific meaning, and the meaning is not so fluid.

How do you express a more complex or different idea, then? It is through the combination of SIGNS which each may represent individual POTENTIAL concepts that I am able to express my thoughts to you. By agreement (and education) we are both aware of the potential meanings that a specific word might carry. Take for example this word (sign):

blue

When I show you that word in this context, what I want you to recognize is that by itself, I am merely describing its “sign”-ness. Those four letters in that combination form a word. That word when placed into context with other words may represent several different and distinct ideas. But by itself, it is all just potential. When you read that word above, you cannot tell if I’m going to mean one of the colors we both might be able to see, or if I might be about to tell you about an emotional state, or if I might describe the nature of the content of a comedian’s act I just saw…

While I can use that sign when I describe to you any of those specific meanings, in and of itself, absent of other symbols or context, it is just a sign with all of those ambiguous, potential meanings, but in the context of our discussion, it has no specific meaning.

It has a form, obviously, and it has been constructed following rules which

Photo of an Actual Stop Sign In Its Normal Context

you and I now tacitly understand. Just as a stop sign has been constructed following rules we have been trained to recognize.

Imagine now a warehouse at the Department of Transportation where a pile of new stop signs has been delivered. Imagine they are laid flat and stacked on a pallet, just waiting to be installed on a corner near you.

While they lay in that stack, they certainly have substance, and they each have the potential to mean something, but until they are placed into a proper context (at a corner by a road) their meaning is just as ambiguous as the word sign above. If you were driving a fork lift through the warehouse and came upon the pallet, would you interpret the sign right then as applying to you? Probably not! Could you say, just be looking at an individual instance of a sign, exactly which cars on which road it is intended to stop? No, of course not.

So this is the distinction between the sign and the meaning of a symbol. The sign is a physical construct. When placed into a recognized context, it represents a specific meaning. In that context, the sign will only carry that one specific meaning. If I make another instance of the sign and put it in a different context, while the signs may look the same, they will not mean the same, and hence I will have made two different symbols.

Just to be perfectly clear on the metaphor I’m presenting, here is a “pile” of signs (words) which I could use in a context to express meaning:

blue

blue blue

blue blue blue blue

Now let me use some of them and you will see that given a context (which in this case consists of other word signs and some typcal interpretations) I express different meanings (the thoughts in your head when you read them together):

once in a blue moon

blue mood

blue sky project

blue eyes crying in the rain

But make no mistake, while i have now expressed several different ideas to you using the same sign in different contexts, they are each, technically, NOT THE SAME SIGN AT ALL! Rather they are four examples of a type of sign, just as each of the stop signs on that pallet at the DoT are examples of a type of sign, but each is uniquely, physically its own sign! This subtlety is I think where a lot of people’s thinking goes awry, leading to conflation and confusion of the set of all instances of a sign with all of the concepts which the SET of signs represents.

To make this easier to see, consider the instance of the word (sign) “blue” above which I have colored red. That is a specific example of the “blue” sign, and it has a specific, concrete meaning which is entirely different from the word (sign) “blue” above which I have colored green.  The fact that both phrases have included a word (sign) of “blue” is almost coincidental, and does not actually change or alter the individual meanings of the two phrases on their own.

Finally, since I have belabored my nit-picking a bit, if I were to re-word your initial statement slightly to use the terminology I prefer on this site, It would change to:

But if you take more [signs], and put them together to get yet another meaning. Don’t you feel like those [signs] were again like symbols creating a new meaning?

And to this question, it should be clear, that my answer is “Yes, precisely: when you put other signs together, you create new meaning”.

Meaning Over Transformation

This entry is probably ahead of the story, but I wanted to start moving into this subject and I’m not yet organized. It should make more sense later on when I’ve explained such things as the “magical” function M() more thoroughly.

Review: The Magical Function “M()”

As a review for those who may not have seen this function previously on this site, I have invented a mysterious and powerful function over all things used as signs by humans. Named the “M()” function, I can apply it to any symbol or set of symbols of any type and it will return what that symbol represents. I call it the “M() function because it takes something which is a symbol and it returns its meaning (that’s all of its meaning).

How Meaning Carries Over Symbol Transformations

When we move information from one data structure to another, we may or may not use a reversible process. By this I mean that sometimes a transformation is a one-way operation because some of the meaning is lost in the transformation. Sometimes this loss is trivial, but sometimes it is crucial. (Alternatively, there can be transformations which actually add meaning through deductive reasoning and projection. SFAT (story for another time))

Whether a transformation loses information or not, there are some interesting conclusions we can illustrate using my magical, mysterious function M(). Imagine a set β of data structure instances (data) in an anchor state. The full meaning of that data can be expressed as M(β). Now imagine a transformation operation T which maps all of the data in β onto a second set of data Δ.

T : β |–> Δ such that for each symbol σ in β, there is a corresponding symbol δ in Δ that represents the same things, and σ <> δ

By definition, since we have defined T to be an identity function over the meaning of β, then we can conclude that if we apply M() before and after the transformation, we will find ourselves with an equivalence of meaning, as follows:

By definition: T(β) = Δ

Hence: M( T(β) ) ≡ M( Δ )

Also, by definition of T(), then M( β )  ≡ M( T(β) )

Finally, we conclude: M( β ) ≡ M( Δ )

Now, obviously this is a trivial example concocted to show the basic idea of M(). Through the manner by which we have defined our scenario, we get an obvious conclusion. There are many instances where our transformation functions will not produce equivalent sets of symbols. When T() does produce an equivalence, we call it a “loss-less” transformation (borrowing a term from information theory) because no information is lost through its operation.

Another relationship we claim can also be defined in this manner is namely that of semantic equivalence.  This should be obvious as well, from reflection, as I was careful above to refer to “equivalence of meaning”, which is really what I mean when I say two things are semantically equivalent. In this situation, we defined T() as an operation over symbols such that one set of symbols were replaced with a different set of symbols, and the individual pairs of symbols were NOT THE SAME (σ <> δ)! In a most practical sense, what is happening is that we are exchanging one kind of data structure (or sign) with another, such that the two symbols are not syntactically equivalent (they have different signs)  but they remain semantically equivalent. (You can see some of my thoughts on semantic and syntactic equivalence by searching entries tagged and/or categorized “equivalence” and “comparability“.)

A quick example might be a data structure holding a person’s name. Let’s say that within β the name is stored as a string of characters in signature order (first name  middle name  last name) such as “John Everett Doe”. This symbol refers to a person by that name, and so if we apply M() to it, we would recognize the meaning of the symbol to be the thought of that person in our head. Now by applying T() to this symbol, we convert it to a symbol in Δ, also constructed from a string data structure, but this time the name components are listed in phone directory order (last name, first name middle name) such as “Doe, John Everett”. Clearly, while the syntactic presentation of the transformed symbol is completely different, the meaning is exactly the same.

T(“John Everett Doe”) = “Doe, John Everett”

M( T(“John Everett Doe”) ) ≡ M( “Doe, John Everett” )

M( “John Everett Doe” ) ≡ M( T(“John Everett Doe”) )

M( “John Everett Doe” ) ≡ M( “Doe, John Everett” )

“John Everett Doe” <> “Doe, John Everett”

When the transformation is loss less, there is a good chance that it is also reversible, that an inverse transformation T ‘ () can be created. As an inverse transformation, we would expect that T ‘ () will convert symbols in Δ back into symbols in β, and that it will also carry the meaning with complete fidelity back onto the symbols of β. Hence, given this expectation, we can make the following statements about T ‘ ():

T ‘ (Δ) = β

M( T ‘ (Δ) ) ≡ M( β )

By definition of T ‘ (), then M( Δ )  ≡ M( T ‘ (Δ) )

And again: M( Δ ) ≡ M( β )

Extending our example a moment, if we apply T ‘ () to our symbol, “Doe, John Everett”, we will get our original symbol “John Everett Doe”.

Meaning Over “Lossy” Transformation

So what happens when our transformation is not loss-less over meaning? Let’s imagine another transformation which transforms all of the symbols σ in β into symbols ε in Ε. Again, we’ll say that σ <> ε, but we’ll also define T ‘ ‘ () as “lossy over meaning” – which just indicates that as the symbols are transformed, some of the meaning of the original symbol is lost in translation. In our evolving notation, this would be stated as follows:

T ‘ ‘ (β) = Ε

M( T ‘ ‘ (β) ) ≡ M( Ε )

However, by the definition of T ‘ ‘ (), then M( β )  !≡ M( T ‘ ‘ (β) )

Therefore: M( β ) !≡ M( Ε )

In this case, while every symbol in β generates a symbol in Ε, the total information content of Ε is less than that in B. Hence, the symbols of the two sets are no longer semantically equivalent. With transformations such as this, the likelihood that there is an inverse transformation that could restore β from Ε becomes more unlikely. Logically, it would seem there could be no circumstances where β could be reconstituted from Ε alone, since otherwise the information would have been carried completely across the transformation. I don’t outright make this conclusion, however, since it depends on the nature of the information lost.

An example of a reversible, lossy transformation would include the substitution of a primary key value for an entire row of other data which in itself does not carry all of the information for which it is a key, but which can be used in an index fashion to recall the full set of data. For example, if we created a key value symbol consisting of a person’s social security number and last name, we could use that as a reference for that person. This reference symbol could be passed as a marker to another context (from β to Ε, say) where it could be interpreted only partially as a reference to a person. But which person and what other attributes are known about that person in the new context Ε if we define the transformation in such a way that all of the symbols for these other attributes stay in β? Not much, making this transformation one where information is “lost” in Ε.  However, due to its construction from β, the key symbol could still be used on the inverse transformation back to β to reconstitute the missing information (presuming β retains it).

An example of a one-way transformation might be one that drops the middle name and last name components from a string containing a name. Hence, T ‘ ‘ ( “John Everett Doe” ) might be defined to result in a new symbol, “John”. Since many other symbols could map to the same target, creating an inverse transformation without using other information becomes impossible.

Types of Information Flow

In a previous post a week or so ago, I riffed on an example of communication between two mountain hikers suggested by Barwise and Seligman (authors of a theory of “information flow”). I made the initial distinction between information flowing within a shared context (in the example, this was the context of Morse Code and flashlight signals), and information flowing from observations of physical phenomenon.
Both types of information movement is covered by Barwise and Seligman’s theory. I propose a further classification of various examples of information flow which will become important as we discuss the operations of individuals across and within bridging contexts.

Types of Flows

Symbols are created within a context for various reasons. There’s a difference between generic information flow and symbollic communication.
Let’s consider a single event whereby information has flowed and been recognized by a person. There are three possible scenarios which may have occurred.

1. Observation/Perception: the person experiences some physical sensation; the conditions of some physical perception leads the person within the context of that perception (and his mental state) to recognize the sensation as significant. In this case, the person recognizes that something has occurred that was important enough to become consciously aware of it’s occurrence. This is new information, but is not necessarily symbollic information.

2. Inference/Deduction: A person within the mental state corresponding to a particular context applies a set of “rules of thumb” over a set of observations (of the first type, likely, but not necessarily exclusively). Drawing on logical inference defined by his current context, he draws a conclusion which follows from these observations to generate new information. This is new information in the sense that without the context to define the rules of inference, those particular perceptions would not have resulted in the “knowledge” of the inference conclusion. They would remain (or they would dissipate) uninterpretted and unrelated forever.

3. Interpretation/Translation: This is the only type of information flow that happens using exclusively symbollic mechanisms. In this type of flow, the person receiving the flow recognizes not only the physical event, but also that the observed phenomenon is symbollic: in other words, that some other person has applied additional meaning to the phenomenon (created a symbol or symbols from the physical media by attaching an additional concept to it). In this type of flow, the perceiving person doesn’t simply register the fact of the physical event, but also recognizes that the physical phenomenon satisfies some context-driven rules of material selection and construction indicating that some other person intentionally constructed it. From this knowledge, the perceiver concludes, assuming they are familiar with the encoding paradigm of the sender’s context, that there is an intended, additional message (meaning) associated with the event. The perceiving party is said to share the context of the sending party if they are also able to interpret/translate the perceived physical sign to recognize the concepts placed there by the sender. In this scenario, the person recieving the message is NOT creating new information. All of the information of this flow was first realized and generated by the message’s sender. (This will be an important detail later as we apply this trichotomy to the operation of software.)

In all three types of information flows, as described by Barwise and Seligman, the flow is dependent on the regularities of the physical world. This regularity requirement applies from the regularity of physical phenomenon, to the reliability of the perceptual apparatus of the perceiver, all the way to the consistency of the encoding paradigm defined by the sender’s context.

Peirce’s Modes of Relationship

According to a terrific survey book on semiotics by Daniel Chandler that I’m reading now, Charles Peirce defined types of signs by whether they were symbollic, iconic, or indexical. If I understand Chandler’s summary, the first two examples of information flow I’ve described are at minimum dependent on Peirce’s indexical signs, alternatively called “natural signs”, because these are the natural perception of reality independent of context. Both the iconic and symbollic signs are only recognizable within a context making both fall under my “interpretation” type of information flow.

For the most part, I will treat the iconic and symbollic signs as the same sort of thing for now.

Example Interaction Between Parent and Child Context

In a previous post, I described in general some of the relationships that could exist between and across a large organization’s sub-contexts. What follows is a short description of some actual observations of how the need for regional autonomy in the examination and collection of taxes affected the use of software data structures at the IRS.

Effect of Context on Systems and Integration Projects

July 15, 2005

Contexts lay claim to individual elements of a syntactic medium. A data structure (syntactic medium) used in more than one context by definition must contain meaningful symbols for each context. Some substructures of the data structure may be purposefully “reserved” for local definition by child contexts. In the larger, shared context, these data structures may have no meaning (see the idea of “traveller” symbols). When used by a child context, the meaning may be idiosyncratic and opaque to the broader context.

One way this might occur is through the agreement across different organizational groups that a certain structure be set aside for such uses. Two examples would include the automated systems at the IRS used respectively for tax examinations and tax collections.

Within the broad context defined by the practitioners of “Tax Examination” which the examination application supports, several child contexts have been purposefully developed corresponding to “regions” of the country. Similar organizational structure have also been defined for “Tax Collection” which the collection application supports. In both systems, portions of the syntactic media have been set aside with the express purpose of allowing the regional contexts to project additional, local meaning into the systems.

While all regions are contained in the larger “Examination” or “Collection” contexts, it was recognized that the sheer size of the respective activities was too great for the IRS central offices to be able to control and react to events on the ground in sufficient time. Hence, recognizing that the smaller regional authorities were in better position to diagnose and adjust their practices, the central authorities each ceded some control. What this allowed was that the regional centers could define customized codes to help them track these local issues, and that each application system would capture and store these local codes without disrupting the overall corporate effort.

Relying on the context defined and controlled by the central authorities would not be practical, and could even stifle innovation in the field. This led directly to the evolution of regional contexts. 

Even though each region shares the same application, and that 80 to 90% – even 95% – of the time, uses it in the same way, each region was permitted to set some of its own business rules. In support of these regional differences in practice, portions of the syntactic medium presented by each of the applications were defined as reserved for use by each region. Often this type of approach would be limited to classification elements or other informational symbols, as opposed to functional markers that would effect the operation of the application.

This strategy permits the activities across the regions to be rolled up into the larger context nearly seamlessly. If each region had been permitted to modify the functionality of the system, the ability to integrate would be quickly eroded, causing the regions to diverge and the regional contexts to share less and less with time. Eventually, such divergence could lead to the need for new bridging contexts, or in the worst case into the collapse of the unified activity of the broader context.

By permitting some regional variation in the meaning and usage of portions of the application systems, the IRS actually strengthened the overall viability of these applications, and mitigated the risk of cultural (and application system) divergence.

Brass Tacks and Comparability

So I thought I should try to explain “comparability” very simply. Reading my previous posts, which were derived from larger texts, I spend a lot of time saying a lot of generalities, and I think the main point is getting missed. So here’s me getting down to brass tacks on the subject.

A computer CPU is a very basic electrical device. Send it a stream of electrons and a command to “add”, and it returns another stream of electrons representing a purely “mechanical” (i.e., unintelligent) electrical result. That CPU doesn’t know anything about semantics, or whether the switches and gates it opens and closes should appropriately be applied to those particular data streams. It just does what it was designed to do given that particular sequence of electron streams. If the streams are comparable before they get to the CPU, then the output will be meaningful. If they are not comparable, then the output (and being a CPU, there will be some output) will not be meaningful.

So the job of the software is to manipulate each symbol before presenting it to the CPU. In particular, the software needs to take each symbol and replace it with one that MEANS the same as the original symbol, but which will present itself to the CPU as COMPARABLE to the other symbols.

Comparability has to be put into the computer, through the software, by a human being. In particular, it is the human who understands when one data stream is not comparable to another, and it is the human being who writes the code to change one stream so that it becomes comparable to the other.

So what really are we talking about? Let me make a non-computer example to show the point.

2 + 00000010 = IV

If I take a pencil and write the above string of characters on a piece of paper, and show it to another computer programmer, after a few moments, I would expect that person to agree that this is a correct mathematical statement

 two plus two equals four

Part of the success of the person in understanding the original statement is that they are able to parse each symbol in the string, interpret the MEANING of each symbol, then translate each into COMPARABLE numeric ideas.

If the computer CPU could experience each symbol as I’ve written it (let’s agree that each of the symbols depicted here would have similar diversity of structure in the computer as they do here on the page), then we can immediately grasp what comparability is. The CPU does not know what the symbols mean, it cannot make the interpretation just by looking at the symbols as they are presented and come to the same conclusion as the human. 

If we look at what I, the human did, to provide you, the reader, with a more readable version of the equation, I replaced each symbol with another one that meant the same, but which appeared as mutually comparable symbols:

  • 2   –>  two
  • +  –>  plus
  • 00000010  –>  two
  • =  –>  equals
  • IV  –>  four

Before the CPU can compare the symbol “2” to the symbol “00000010”, they must both be replaced with two other symbols, each with the standard interpretation of “two”. These new symbols must be structured to flow through the CPU in such a way that their very structure is modified by the CPU to create a third symbol whose standard interpretation has the meaning “four”. The “plus” symbol must be translated into the CPU’s “ADD” instruction, and the “equals” symbol is represented by the stream of electricity leaving the CPU with the resulting symbol.

Why Comparability Is Critical To Solving The Data Integration Problem

At its most basic, the task of data integration from multiple source systems is one of recognizing the EQUIVALENCY and diagnosing the CONFLICTS among sets of symbols (the data) stored in each system’s data structures (syntactic media). Data integration is accomplished when the conflicts have been eliminated through TRANSFORMATION into new COMMON SYMBOLS which are COMPARABLE at both the syntactic and semantic levels.

The end result of data integration should be that SEMANTICALLY EQUIVALENT (or at least COMPARABLE) data structures become SYNTACTICALLY EQUIVALENT (COMPARABLE) as well. When this result is achieved, the data structures are considered COMPARABLY EQUIVALENT, and the data from the different source systems can be collapsed, combined or integrated correctly.

Structural Comparability

The issue can be characterized as one of the COMPARABILITY of data between systems.

  • Syntactic Comparability is defined by the DATA TYPE and internal DATA STRUCTURE
  • Semantic Comparability is defined by the CONCEPT or MEANING projected onto the data structure by the users of the source system
  • Two data items are COMPARABLE if they share both SYNTACTIC and SEMANTIC COMPARABILITY

Typical Conflicts

Typical conflicts occur between and among the data structures originating from different sources.

  • Syntactic Conflicts:
    • Data Type Conflicts
    • Structural Conflicts
    • Key Conflicts
  • Semantic Conflicts:
    • Scale Conflicts
    • Abstraction/Formula Conflicts
    • Domain Conflicts
  • Symbol Conflicts:
    • Naming Conflicts (Synonyms, Homonyms, Antonyms)

Syntactic Conflicts

  • Data Type Conflicts – The same concept projected onto different physical representations. Example: different codes for the same set of options
  • Structural Conflicts – For example, the same concept (referent) represented in one database by only a single attribute in one data source, but as a complete record of attributes in another source.
  • Key Conflicts – Two systems using different unique keys for the same concept.
    • As an example, from a freight rail project I once worked, one set of systems represented a “station” by using the nearest Mileboard number to the station, while another set used an industry standard designator called a “SPLC” which was a code assigned to every reported station on all rail lines in North America.
    • In this example, the two different keys conflicted syntactically (e.g., Mileboard was an integer, SPLC was a string), and semantically (e.g., Mileboards are only meaningful within the context of a single railroad, being the distance from the origin of the line, while SPLCs are universal designators within the context of North America railroads).

Semantic Conflicts

  • Scale Conflicts
    • Same data structure but representing different units. For example, corporate revenue represented as currency, but one using US Dollars and the other using CANADIAN Dollars.
  • Abstraction/Formula Conflicts
    • Same data structure and “symbol”, but two different formulas used to calculate values.
  • Domain Conflicts
    • Similar symbols and data structure, but two different sets of valid values or ranges of values.
    • For example, references to Customers in two systems each have assigned numeric identifiers, but the same customer has different assigned identifiers in each system.

Data Integration

The data integration specification documents how the symbols in two (or more) systems are similar and how they are different. The specification describes how the conflicts identified (under the rough categories described above) can be resolved to produce and combine comparable data symbols from each system. From a practical point of view, researching and documenting/describing the conflicts and similarities between symbols in two different systems is the same activity as defining the data integration specification which would be used to automate the integration.

Functions On Symbols

Data integration is a complex problem with many facets. From a semiotic point of view, quite a lot of human cognitive and communicative processing capabilities is involved in the resolution. This post is entering the discussion at a point where a number of necessary terms and concepts have not yet been described on this site. Stay tuned, as I will begin to flesh out these related ideas.

You may also find one of my permanent pages on functions to be helpful.

A Symbol Is Constructed

Recall that we are building tautologies showing equivalence of symbols. Recall that symbols are made up of both signs and concepts.

If we consider a symbol as an OBJECT, we can diagram it using a Unified Modeling Language (UML) notation. Here is a UML Class diagram of the “Symbol” class.

UML Diagram of the "Symbol" Object

UML Diagram of the "Symbol" Object

The figure above depicts how a symbol is constructed from both a set of “signs” and a set of “concepts“. The sign is the arrangement of physical properties and/or objects following an “encoding paradigm” defined by the members of a context. The “concept” is really the meaning which that same set of people (context) has projected onto the symbol. When meaning is projected onto a physical sign, then a symbol is constructed.

Functions Impact Both Structure and Meaning

Symbols within running software are constructed from physical arrangements of electronic components and the electrical and magnetic (and optical) properties of physical matter at various locations (this will be explained in more depth later). The particular arrangement and convention of construction of the sign portion of the symbol defines the syntactic media of the symbol.

Within a context, especially within the software used by that context, the same concept may be projected onto many different symbols of different physical media. To understand what happens, let’s follow an example. Let’s begin with a computer user who wants to create a symbol within a particular piece of software.

Using a mechanical device, the human user selects a button representing the desired symbol and presses it. This event is recognized by the device which generates the new instance of the symbol using its own syntactic medium, which is the pulse of current on a closed electrical circuit on a particular wire. When the symbol is placed in long term storage, it may appear as a particular arrangement of microscopic magnetic fields of various polarities in a particular location on a semi-metalic substrate. When the symbol is in the computer’s memory, it may appear as a set of voltages on various microscopic wires. Finally, when the symbol is projected onto the computer monitor for human presentation, it forms a pattern of phosphoresence against a contrasting background allowing the user to perceive it visually.

Note through all of the last paragraph, I did not mention anything about what the symbol means! The question arises, in this sequence of events, how does the meaning of the symbol get carried from the human, through all of the various physical representations within the computer, and then back out to the human again?

First of all, let’s be clear, that at any particular moment, the symbol that the human user wanted to create through his actions actually becomes several symbols – one symbol for each different syntactic representation (syntactic media) required for it to exist in each of the environments described. Some of these symbols have very short lives, while others have longer lives.

So the meaning projected onto the computer’s keyboard by the human:

  • becomes a symbol in the keyboard,
  • is then transformed into a different symbol in the running hardware and operating system,
  • is transformed into a symbol for storage on the computer’s hard drive, and
  • is also transformed into an image which the human perceives as the shape of the symbol he selected on the keyboard.

But the symbol is not actually “transforming” in the computer, at least in the conventional notion of a thing changing morphology. Instead, the primary operation of the computer is to create a series of new symbols in each of the required syntactic media described, and to discard each of the old symbols in turn.

It does this trick by applying various “functions” to the symbols. These functions may affect both the structure (syntactic media) of the symbol, but possibly also the meaning itself. Most of the time, as the symbol is copied and transferred from one form to another, the meaning does not change. Most of the functions built into the hardware making up the “human-computer interface” (HCI) are “identity” functions, transferring the originally projected concept from one syntactic media form to another. If this were not so, if the symbol printed on the key I press is not the symbol I see on the screen after the computer has “transformed” it from keyboard to wire to hard drive to wire to monitor screen, then I would expect that the computer was broken or faulty, and I would cease to use it.

Sometimes, it is necessary/desirable that the computer apply a function (or a set of functions called a “derivation“) which actually alters the meaning of one symbol (concept), creating a new symbol with a different meaning (and possibly a different structure, too).

Tension and Intention: Shifting Meaning in Software

If a software system is designed for one particular purpose, the data structures will have one set of intended meanings. These will be the meanings which the software developers anticipated would be needed by the user community for which the software was built. This set of intended meanings and the structure and supported relationships make up the “domain” of the software application.

When the software is actually put to use, the user community may actually redefine the meaning of certain parts of the syntactic media defined by the developers. This often happens at the edges of a system, where there may exist sets of symbols whose content are not critical to the operating logic of the application, but which are of the right syntactic media to support the new meaning. The meaning that the user community projects onto the software’s syntactic media forms the context within which the application is used. (See “Packaged Apps Built in Domains But Used in Contexts“)

Software typically has two equally important components. One is the capture, storage, retreival and presentation of symbols meaningful to a human community. The second is a set of symbol transformation processes (i.e., programming logic) which are built in to systematically change both the structure and possibly the meaning of one set of symbols into another set of symbols.

For a simplistic example, perhaps the software reads a symbol representing a letter of the alphabet and it transforms it into an integer using some regular logic (as opposed to picking something at random). This sort of transformation occurs a lot in encryption applications, and is a kind of transformation which preserves the meaning of the original symbol although changing completely its sign (syntactic medium).

When we push data (symbols) from a different source or context into the software application, especially data defined in a context entirely removed from that in which the software was defined and currently used, there are a number of possible ways to interpret what has happened to the original meaning of the symbols in the application.

What are some of the ways of re-interpretation?

  1. The meaning of the original context has expanded to a new, broader, possibly more abstract level, encompassing the meanings of both the original and the new contexts.
  2. Possibly, the mere fact that the original data and the new data have been able to be mixed into the same syntactic media may indicate that the data from the two contexts are actually the same. How might you tell?
  3. Might it also imply that the syntactic medium is more broadly useful, or that the transformation logic are somewhat generically applicable (and hence more semantically benign)?
  4. Are the data from the two contexts cohabitating the same space easily? Are they therefore examples of special cases of a larger, or broader symbollic phenomenon, or merely a happy coincidence made possibe by loose or incomplete software development practices?
  5. How do the combined populations of data symbols fare as maintenance of the software for one of the contexts using it is applied? Does the other context’s data begin to be corrupted? Or is it easy to make software changes to the shared structures? Do changes in the logic and structure supporting one context force additional changes to be made to disambiguate the symbols from the other context?

These questions come to mind (or should) whenever a community starts thinking about using existing applications in new contexts.

%d bloggers like this: