I have been asked on a couple occasions about how to do this, and I get the impression that the answer I give doesn’t satisfy the people asking, because I basically tell them, “You’re trying to do it wrong.” I try as succinctly as I can to give them some ideas on how to do it, but I get the feeling they hear me like I’m an alien, because it’s not what they expected.
Abstract data types are not OOP. I know that message is confusing, because that’s how I felt when I heard it. I thought that’s what objects were, because that’s basically what I was told for years, by multiple sources that I thought knew what they were talking about.
For those who need some catch-up on this, I wrote a post a while back on the subject: A collection of (Quora) answers on object-orientation.
Those who understand Smalltalk might think this subject is obvious, because Smalltalk is built from non-OOP stuff. The point of what I’m talking about is I hope to address those who don’t want to rebuild Smalltalk, but want its architectural advantages.
A project I’ve heard Alan Kay reference a couple times is one he consulted on more than three decades ago, which he’s called Brooklyn Union Gas. The name of it was CRIS-II. It was an enterprise system that Brooklyn Union Gas designed and built themselves, with help from Alan, while he was working with Andersen Consulting, and they built it on the model of Smalltalk. However, they built it completely using standard tools that were on IBM mainframes at the time. If this needs to be said more explicitly, they didn’t create an OO language. Instead, they created an OO architecture from these standard system tools. Their system used all of the facilities: operating system, a programming language, an RDBMS, and transaction management as the programming environment upon which the application logic was based. Each tool was used as given by IBM, by which I mean to say it does not seem to me they “opened them up,” and modified them. Instead, they used them in a novel way.
If you look online, you can find some biographical references to articles on this project, but probably the best you’re going to get is behind a paywall. I managed to find an article on it in the publication IIIE Software, from January 1993, through interlibrary loan. It’s titled “Object-Oriented Development at Brooklyn Union Gas.” I’m not going to reproduce the full text here (since that would be illegal), but I will summarize what I think were its salient points. The article does not give in technical detail how to do what they did, but it provides some strong hints.
“Object-Oriented Development at Brooklyn Union Gas,” by John Davis of Andersen Consulting, and Tom Morgan of Brooklyn Union Gas, IEEE Software, January 1993
Using object-oriented features, a mainframe implementation of a Smalltalk-like execution environment supports a critical application and can accommodate change.
The CRIS-II architecture is structured in three layers that are derived from the essential aspects of the underlying problem domain. The interface, process, and business-object layers address, respectively, the “when,” “what,” and “how” in the gas utility environment.
What’s in the Interface layer:
- UI
- batch processes
- integration with other applications (not designed using this OOP scheme)
- UI is not always composed of objects; has access to objects via. message-passing.
- Events which trigger functions in the process layer.
The Process layer:
- Is built out of “function managers”: Carries out response to events received from the interface layer.
- Is described by a script-like control structure, which drives the system’s response to events.
- The script is implemented using message sends to objects in the business-object layer.
- The function-manager script describes “what” to do. Delegates “how” to the business-object layer.
- The script sequences the use of business objects
Business-object layer:
- Receives messages from the Interface and Process layers.
- The “how” of this layer has sublevels. For example:
Successive answers to the “how do you…?” questions produce layers of functions that produce additional layers of functions within the business-object layer, beginning with the business policy and practice and ending with the technical details of the current implementation.
System tools used to build and operate CRIS-II:
- IBM mainframe
- MVS/ESA operating system
- DB2 database manager
- CICS (Customer Inquiry and Control System) online transaction manager
- PL/I programming language
The overall system design borrowed many architectural concepts from Smalltalk-80 and Brad Cox’s work on Objective C.
Some technical details:
The system allowed untyped variables, used dynamic messaging (messages formulated at run-time), used encapsulated objects, and provided some metaprogramming, so that class information could be called up at run-time.
Characteristics
As in traditional OOP, an object in this system was an instance of a class, and implemented inheritance. All objects were descendants of Object.
Classes had both class and instance methods.
Methods were written in PL/I, and each method was its own separate executable. Methods were bound at run-time. They were known in the system by their selector name.
Classes were defined, and methods/behaviors were associated with classes in dictionaries that were laid out in entity-relation-attribute sequences. The dictionaries were loaded into memory on demand, where they could be accessed as objects.
As such, instance variables could be changed without recompiling any methods.
Objects communicated via. message passing.
Dynamic messaging (formulated at run-time, like “perform” in Smalltalk) was supported, and used frequently.
Objects used “self” and “super” tokens to send messages to themselves, and to their superclass, respectively.
Non-OO entities used were strings, numbers, and characters, which were stored as native PL/I data types.
Object model
Since CRIS-II was designed as a multi-user system, objects for each user were stored in single-thread sessions. Each thread had a Context object, which was a container housing all of the objects currently in use for a user’s session. They implemented methods for managing storage (storage allocation, synchronization, and serialization, and storage reclamation).
Context objects:
- could find every object they stored by class.
- did not implement garbage collection.
- stored objects in fixed-length chunks, which optimized performance, since the storage space could be reused by different objects.
- enabled transaction management/versioning behavior
The overall object system was largely implemented in itself, but to improve performance, object storage was implemented in PL/I.
During execution, new objects were stored in a Context object. Storage grew and shrank, depending on how many objects were being used. At times, Context objects would close gaps in their management of memory by physically moving objects into contiguous cells.
Transaction management
New objects tended to be initialized with data stored in the DB2 database.
Before initializing a new object, object persistence behaviors checked the Context object to make sure the same instance was not already present. Each instance had a unique identifier. This allowed applications to insure that the most recent versions of instances were returned.
When a new instance was placed in a Context, its state (instance variables) was saved in the database. This optimized update processes when object state was checked back into the database manager.
The system could create detailed audit trails in a standardized way. Control behaviors in the system often set up “before” and “after” values of instance variables in special Activity objects.
In many cases, application exceptions and side-effects were created when certain variables changed. An example that was used was when a service order status change might need to be transmitted to other systems. Identifying changes in a standardized way let the system abstract side-effect and exception handling into a superclass. This helped maintainers, if they needed to change the implementation of side-effects.
Multiple instances of a single user’s Context could exist. By saving Contexts at checkpoints, and reloading them, the system could rewind processes back to earlier states. A thread could also use this to pursue alternate execution paths.
The interface (UI) layer controlled the logical unit of work. When a business event completed, the interface informed its Context object. It then iterated over its encapsulated objects, sending “save” messages. Objects that had changed state serialized their state to the database.
The Context object interacted with the database in a sequence that optimized database response, and avoided deadlocks. The Context object carried out the implications of side-effects for events, and made sure current and saved states were synchronized.
When the end of a logical unit of work was reached (for example, when a company representative completed a telephone call with a customer), the Context object was sent a “clear” message to erase most of its objects and reclaim their storage space. Some objects, such as those that identified the current thread and its user, would only respond to a “purge” message.
This scheme created a rudimentary garbage collection system. It was also the largest source of bugs in the system. The authors said a real garbage collector “would have been invaluable.”
Database-manager interface
Each business-object usually had a dedicated row in a database table. However, an instance of an active object contained instance variables that represented the object’s whole-part structure. Extensions to accessors let clients of objects retrieve initialization values, even if a logical unit of work had progressed past that state.
Persistent objects were descended from an abstract class that implemented data access behaviors using SQL.
The database-manager interface was a sublayer of the business-object layer, separating application concerns from the technical details of implementing actions. This allowed easy substitution of alternate implementations.
Object navigation
Navigational messages were used to travel around the whole-part structure of an object. Each object understood its component parts and implemented selectors for returning them.
In the database, foreign keys pointed to the parent in a whole-part relation, and were accessed using SQL statements. Each object included a method for navigating to its parent, or “whole.”
CRIS-II implemented a kind of “lazy evaluation.” When an object was initialized, all of its instance variables were initialized to null. The first time a request was made for its contents, only then was it initialized with actual values.
This behavior assembled, from encapsulated objects, the needed states. It reduced coupling, and eliminated most of the data-access code.
Why Brooklyn Union Gas needed an object-oriented system
The company’s customer information system managed field-service orders, cash processing, credit and collections, meter reading, billing and general accounting functions. Each year, the system processed one-billion-dollars in annual revenues.
The company needed to replace a system that had been implemented in the 1970s. They planned to use a data-driven design with integrated CASE tools. However, this seemed to lead to the same issues as the system they were trying to replace. The complexity grew larger than they thought. They decided on a different approach, that such a system is best thought of as a collection of simulations of the business, and its environment.
Development of their new system took place between 1987 and 1989, involving Brooklyn Union Gas and Andersen Consulting employees.
Type system
The system permitted, but didn’t require types to be specified for method variables. Explicit types improved performance, but though the developers made explicit measurements of the frequency of program errors during development, they did not find that this practice reduced the number of logic, or resource access/allocation errors in the system.
With explicit types, a program may by correct, if and only if the values present in the variable are of the specified type. This condition came at the cost of design/maintenance flexibility. Nothing in the team’s measurements of the development process showed that explicit types made error-detection efficient. Rather, they found the inflexibility on design this practice imposed was harmful. A better approach was to look at types as one of many assertions they could make about variables, at certain phases of the development cycle, and one of a variety of techniques the team could use to ensure efficient debugging.
The Development Environment
The detail and particularity necessary in object-oriented applications, and the system’s scale made an extensive array of development tools essential. The authors said, “You cannot do object-oriented development on this scale with just a compiler and an editor.” Early on, they constructed their development tools based on traditional mainframe system tools, including IBM’s ISPF (Interactive System Productivity Facility – Provided what would now be called a “dashboard” interface for manipulating data sets, and program modules), DB2, and PL/I preprocessor. Other development software included a “screen painter” (which I take to mean a screen/form layout tool), a code generator, and a report-generation tool.
All of these tools worked with an expandable entity-relation-attribute dictionary that contained information about the applications that made up CRIS-II, and its environment. This central dictionary stored and managed all components for the system.
Re. the visual object environment, it provided means to browse object descriptions and methods. It provided cross-referencing of messages, so a developer could locate all methods that were senders or receivers of a message.
Often, members of the team located behaviors using a cross-class browser, which would find behaviors by partial matches of class and method names. They found this more useful than just being able to look for a class name in the hierarchy.
All team members had access to the dictionary, and used it to record design information, and changes to it. Once the system was installed into production, the dictionary continued to be used during maintenance cycles.
The dictionary was designed to maintain relational links between the interface components and the implementation components that processed event information. It also tracked system test scenarios, and the scripts to execute them. It supported code generation and testing tools, and the configuration of object behaviors.
Their experience was that only a small core of the development team needed extensive training in the structure of this object-oriented infrastructure, and methods of building and maintaining objects. Most of their application developers worked in frameworks where they didn’t have to understand these techniques completely.
Performance and reuse
They found that though message-passing is not as performant as procedure calls, the value of this structure far outweighed the costs. As Hoare would say, “Premature optimization is the root of all evil.” They found that putting too much emphasis on low-level optimization actually decreased the system’s overall performance. As an example, late-bound messaging led to better database-access performance, which created an overall performance gain.
They believed that performance assessments should include the consideration that maintaining tightly-coupled systems creates greater complexity in design, and all the costs that go with that, which overtakes the performance benefits of procedure invocation.
They also found that using their object scheme resulted in greater component reuse than was typical in business systems.
Lines of code by layer
They laid out the breakdown of how much code was in each functional layer:
- Interface layer: 27%
- Process layer: 5%
- Business object layer: 65%
Business objects represented more than half of the total. The largest business object contained no more than 6% of the total code, showing that the functionality was well-distributed.
The process layer metric showed that this architecture let the application say in a small amount of code “what” was to be done.
They described what they called their “hybrid” interface layer (part object-based, part application-based UI) as “bulky,” but the average component size was 100 lines of code. The layer had more than 1,000 components. It was built using integrated CASE tools, which tended to generate duplicate code in each component. They thought this layer would’ve been much smaller had the components been implemented as objects that could get code reuse through inheritance.
They said it’s good to think of systems like CRIS-II like “small economies, rather than large computer programs.” In other words, an evolving system that’s built through experience.
They further said,
Such systems are so large, they actually alter their environment.
System developers must recognize that specifying more than a few steps ahead in this trajectory is simply impossible. The essential requirement for business systems is orderly evolution over time, with minimal constraints on the design trajectory.
If the system models the real environment, and the technical artifacts it needs are well insulated from each other, changes to the system are likely to exhibit ‘proportionate effort’ characteristics: Things easy to ask for will be easy to do.
They extended their entity-relation-attribute dictionary to include design items, and then developed tools that operated on that data, but said they thought it would’ve been better to use the object system itself to carry out what the dictionary did, “Our experience shows enormous benefit from having fully bootstrapped environments.”
The class description, inheritance, and messaging mechanisms were built within their object system, but they were implemented outside of PL/I.
They recommended that if object-oriented extensions are desirable in the languages that exist that programmers should think about constructing them using a language-neutral tool or system that would describe the class structure and messaging mechanism, and that they should limit language extensions to bindings to class descriptions, method implementation, and invocation. “The Art of the Metaobject Protocol,” by Kiczales, Rivières, and Bobrow shows how that might be done, as does the System Object Model described in “OS/2 Technical Library System Object Model Guide and Reference.”
Another source cited in the article, that talked about this system was “Brooklyn Union Gas: OOPS on Big Iron,” in Harvard Business School case study N9-192-144, Boston, 1991.
This is one of a series of “bread crumb” articles I’ve written. To see more like this, go to the Bread Crumbs page.