Railroads, Trains and Rail Cars: Effect of Context on Software Integration

A Concrete Example. August 22, 2004

When I first started working on the railroad data integration project, I began with a naïve notion that the railroad business would be most fundamentally interested in “trains”. This notion came from my own experiences with rail, as a passenger catching and riding trains up and down the East Coast. The context I began with had also been reinforced by my memories of my brother’s model railroad layout. He would play for hours with the thing, moving cars from one switchyard to another, while I only cared about it when the train was actually running around the set. The context I was familiar with was that shared by passengers on other modes of transportation. Like buses and airplanes, there was a pre-scheduled trip on a named route, flight or line.

My customer, however, was in the freight business, which they were quick to point out, could almost care less about specific trains. They explained to me that the fundamental unit of interest to them was the “car”. Their knowledge about cars was extensive and detailed. Were they loaded or empty? Who had the car and where was it? When was it coming back? What characteristics and features did the car have? These matters consumed many of their systems and business practices. Only the workers in the yard really cared about specific “trains”, because they were responsible for constructing an optimal mix of “cars” based on customer contents, delivery priorities, destinations, “power” and direction.

Once I understood this, I was quickly able to orient myself within the data models of their many systems by finding the data structure representing the rail car, then branching out from there. The overall context within which all of their most important computer systems had been built was based on this focus on the car. Their business model was completely dependent upon the car. Revenue was generated by filling the car with freight and moving it down the line. Once the car left their tracks, additional revenue was generated through the use of the car by other rail businesses. Accounting systems, shipment tracking systems, maintenance systems, ordering and fulfillment systems, were all built and configured under the same overarching, and well-defined constraints, upon which there was much agreement within the freight rail community.

Some of the basic notions were so ingrained as basic assumptions in the community that they almost disappeared in regular conversation. Some of these were stated and others not stated until pressed, and they included the following:

  1. The basic unit of information was the “CAR”.
  2. The “CAR” was either full or empty, outbound or inbound. A “SHIPMENT” corresponded directly with a “LOADED CAR”.
  3. The CAR’s state could change (under normal circumstances) from EMPTY to LOADED or vice-versa only at designated locations on the rail line which corresponded with “FACILITIES” where CARS could be loaded or emptied. Shipments would originate and be delivered at these FACIITIES.
  4. These FACILITIES represented a finite set of delivery origin and destination points, were typically semi-permanent, and fixed to a particular location on the track, which was always precisely known before the shipment was ordered.
  5. Shipments (in other terms “CAR LOADS”) could only be picked up from or delivered to one FACILITY.

Over the course of the analysis effort, I discovered that the railroad (and in fact the industry as a whole) had been struggling in recent years with significant competition from the trucking business. The smaller loads and direct, point-to-point service capabilities of trucks offered significant competitive advantages, which led to reduced business for the railroads.

Over the course of several decades, the railroads had attempted several strategies to counter the threat trucking presented. One strategy was the invention of a line of business called “intermodal transport”. The intermodal transport business combined the large-load, long-haul efficiencies of rail, with the flexibility of trucking for the “last mile” of delivery.

As part of this new business approach, specialized containers were invented which would contain the actual shipments, and new rail cars (and other transportation devices, such as container ships and loading docks at ports) were created to carry the containers from one regional distribution center to another. These containers could then be loaded onto trucks and delivered to the loading bay doors of the ultimate customer, no matter where they were on the planet.

As promising as this new type of service was, it had been creating tremendous havoc in the executive suites and back offices of the major railroads. In large part, this was because the new approach broke so many of the assumptions of the freight rail business. The terminology and concepts that all of the front-line and back office computer systems had for years taken for granted would have to change.

One of the fundamental reasons for the difficulty was that the early railroad database designers had relied on the assumptions of the freight rail context, and the introduction of the intermodal business context directly conflicted with these long-standing simplifications, as the following suggests.

  1. The basic unit of intermodal information now was the “CONTAINER” ( a smaller unit than a “CAR”).
  2. The “CONTAINER” might be loaded or empty. It could also contain multiple, partial loads, each representing their own SHIPMENTS. When the CONTAINER was on top of the rail car, what was the status of the CAR (that typically carries more than one container)? Was the CAR empty if the containers were empty? What was the CAR carrying if the containers were loaded? A single CAR might now carry more than one intermodal SHIPMENT at a time.
  3. While the state of the CAR was still changed at a rail FACILITY, the actual origin and destination points of the SHIPMENT could no longer be considered as merely the rail FACILITIES.
  4. The set of origin and destination points for SHIPMENTS was no longer finite. In fact, because both the origin and destination points for intermodal freight could be anywhere on the planet, the actual locations required much more information to be captured.

Many of the original railroad computer systems were built with the equivalence between a shipment and a loaded railcar as a given. In the early days of railroad automation, this notion was so obviously true that many systems did not differentiate between the car and the shipment in any way.

Since humans love analogy, and the best practices of computer system development called for economy of data structure, the substitution of “LOADED RAIL CAR” for “SHIPMENT” was an obvious one to make.

With the introduction of intermodal, this fundamental metaphor was shown to be completely broken. Intermodal shipments require at least three different structures: the “partial load”, the container, and the rail car (for the rail portion). Asking the basic question “where is my shipment” was relatively straight-forward in the purely rail shipment context. Wherever the car was, there was the shipment.

When the railroads changed to accommodate the intermodal business, they found themselves with several software problems. Already mentioned was how to define the state of a rail car carrying containers. Another area of difficulty arose in the tracking of a shipment. Clearly, the railroad had a long history of tracking delivery times based on precisely measured routes and historical travel times. Intermodal changed the meaning of “location” and “route”. Measuring and predicting delivery times for railcars is much easier than for trucks. Rail cars can not leave the track. They can not be loaded or emptied off the track. When shipments were only on rail cars, they were always somewhere along a fixed path.

With intermodal, a shipment could leave known routes and locations and suddenly travel to any place with an address. Rail computer systems were not designed to handle the sheer magnitude of locations. Plus, routes, including the true origin and destination and all intermediate points in between, could no longer be captured with the old “milepost” and “facility” notations.

The solution taken by railroads who moved into intermodal business was either to re-write their original portfolio of systems, or to create/acquire new dedicated systems for tracking intermodal shipments. Later, when management required better information, the original rail systems and the new intermodal systems needed to be integrated. In some cases, until the integration was required, the enterprise business context did not realize that the concepts of SHIPMENT and CAR had to be split apart from each other.

This example shows many typical aspects for all businesses. It presents a good case study for how organizations can suddenly find themselves with data integration as a major problem. The following points should be highlighted:

  1. Using analogy and metaphor, organizations pick out certain details to stand-in for complex concepts (for example the correspondence between a loaded rail car and a shipment permitted one to be used as a metaphor for the other).
  2. Business pressures cause the introduction of a whole new class of services to customers (for example, the threat of trucking led to the invention of intermodal freight).
  3. This new class of services have their own unique exigencies driving the introduction of new data structure needs.
  4. When the business changes, old analogies can be broken by changes in what once appeared as fundamental truths.
  5. While similarities exist between the original business practice and the new, there are often fundamental incompatibilities reflected as well.
  6. While it may be possible to conceive of a reinterpretation of the old analogies in light of the new reality, it may not be practical or possible to modify all of the old systems to accommodate this new vision.
  7. Business must not be impeded, so expediency is always the right choice. This means that it is often the best choice to do what gets the job done fastest, and this often leads to new application acquisition, and the rise of the data integration problem.

Rail industry experts will notice how simplistically I’ve presented their domain. The emphasis of this document is on the larger lessons to be learned from the example, and not a critique of the state of automation in the rail industry. While I’m pointing out various issues of data representation that arose in the rail industry, this is not to imply that these were the biggest or hardest ones the industry has had to overcome.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: