Introducing SimpleBus

SimpleBus is an architectural solution for medium to large scale software design. It is an attempt to limit the growth of complexity in larger software systems. SimpleBus is designed to be implementable in any environment and therefore is limited to a lowest common denominator definition. This is a design feature - not a bug.

One of many problems with large scale software is the management of abstractions. Humans cannot reason about complex systems without resorting to gross generalizations or abstractions. The problem with these generalizations is that they gloss over the implementation details they abstract away, but these details aren't gone - just hidden. If you want to use the abstractions safely, you will still have to understand what it is that is abstracted away. Most abstractions aren't perfect but leaky.

By defining the SimpleBus architecture in an intentionally stupid way it can be implemented as a perfect abstraction in almost any language for any communication medium. By not specifying that the bus will always relay your messages to all listeners, the bus can be implemented on top of imperfect communications channels without the need for an imperfect abstraction layer. The perceived problem is the added complexity for each participant on the bus to accept and work with the fact that delivery is not guaranteed. But in fact this is a bonus. The system as a whole is robust in any of its parts, because any component must be able to handle communications failures. This also means that the system is very scalable and flexible, you can at any time rewire the communication architecture of the entire system - in depth - to scale as needed.

So the rationale for the SimpleBus design is that it must be implementable on any kind of communications medium, be it a single threaded process, threads, sockets or even ip over avian carriers. It must also communicate in a way that can be parsed in any language in very simple systems. Not only that, the parser and bus design should be easily implementable by non-professional programmers. There should be no good reason not to implement the bus - or not to implement it correctly.

One of the things explicitly not in the SimpleBus design is a syntactic or semantic format for the messages. The bus can be used at any level of abstraction in a software system. It doesn't make sense to specify some kind of taxonomy or syntax for all levels of abstraction and I doubt it could be done. The problem this creates is also easily solved because of the simplicity of the design. If you want to use a third-party component that expects a different message syntax or format, you can very easily write a translator component that encapsulates the third party component.

The creation of a common message format for interoperability is not something to avoid, it is just not in the scope of the SimpleBus design. It is something to be built seperate on top of this design - for a specific level of abstraction.

The one thing the SimpleBus design forces on the message format is that the message must contain all the information necessary for a component to work with. This means you cannot send instantiated objects or opened streams. This would mean that there is still a shared state. Something that is communicated between two components through a different channel than the bus. By using such a message the system would break the moment you seperate the components physically over multiple systems.

The SimpleBus design specifically does not specify that the order in which messages are sent is the order in which messages are received. In a multithread or multiproces implementation some operations will take longer than others, failures may occur, etc. A participating component must therefor not assume any specific order in delivery of messages. If any ordering is required, the information on which to base that order must be included in the message data and the listener should compose the messages in the correct order itself.

In fact a SimpleBus design does not even specify that it is reliable. Messages may not be sent to subscribers. It does specify that any message that is delivered must be delivered in whole. Messages must not be corrupted. It is relatively easy to design a derivative of the SimpleBus that does provide reliable message delivery, but this makes it more difficult to scale this bus. It should be avoided at any level where communication over potentially unreliable channels is a foreseeable future requirement - or you will once again have a leaky abstraction.

A SimpleBus is inherently public. Any message sent over the bus is receivable by all listeners. There can be no private communication channel. You can implement adressed messages on top of the bus, but any component may still listen in - even if it is not the addressee. This is a benefit for any system which is expected to change during its usefull lifetime. All information at any specific level of abstraction is available to any component interested in it, there are no back channels.

There is one exception to the above rule - no message is sent to a listener if that listener is the same component that sent the message. If this exception was not there each component must take care that messages sent do not match messages it listens for - or if they do that the component knows it is the originator. This would add unneeded complexity to each component.

A SimpleBus is fundamentally a publish/subscribe system with content-based filtering. The only information sent over the bus is content, so there can be no other filtering. You can define a message syntax that adds an envelope with additional information, e.g. topics, channels, type, etc. The specific filtering method is not specified but is implementation dependant, just like the message format. For simplicity, optimizability as well as ability to reason about the system a declarative filtering system should be preferred. e.g. For an xml based message format xpath would make an obvious filtering language. Similarly for a json based message format an obvious choice is jsonPath.

An important idea in the use of SimpleBus is that it is intended to be used at different levels of abstraction. It is meant to be used in an encapsulated form. You should be able to create ever more complex systems by combining simpler subsystems through a higher level SimpleBus. This does not mean that SimpleBus should be used at every level of abstraction, once you partition the problem domain to a sufficiently small subdomain or a sufficiently low level of abstraction, there is no need for the overhead and added complexity of a SimpleBus. e.g. When you are parsing a single xml document it makes no sense to divide the proces into seperate parts and communicate through a bus.

Because a complex SimpleBus-based system shares no state between different levels of abstraction, you can examine any part of the system without having to understand more of the underlying or overarching levels than is passed on in the messages. There can be no more requirements for any subsystem to function correctly than that the containing system sends correct messages and parses the messages from the subsystem correctly.

Applying al this a SimpleBus implementation must

  1. Be public only - no private channels
  2. Be semantically or factually asynchronous
  3. Allow only simple values in messages - no objects, streams, etc.
  4. Allow an unlimited number of instances of the bus to exist

A SimpleBus implementation has the following inherent qualities:

  • Parallizable
  • Scalable
  • Extensible
  • Polymorphic
  • Error resistant in a granular fashion
  • Fractally similar at every level
  • Testable at any level
  • Potentially changeable without the need for a restart / reboot in the whole

The intention behind the SimpleBus design is to create a design filosofy that allows software to be designed based on succesively smaller and simpler components and that is inherently changeable. It is based on ideas about scale found in 'Patterns of software'  by Richard Gabriel, ideas about the power of simplicity by Alan Kay - inventor of smalltalk - and Dave Winer - inventor of RSS. As well as the solution to complexity taken by functional programming - removing the possibility of shared state by removing the concept of variables entirely.

May 9th
Auke van Slooten

blog comments powered by Disqus