Paul Graham has, as usual, an excellent article on the art of software development: Holding a Program in One’s Head

I have only one question. In his article he writes:

You can magnify the effect of a powerful language by using a style called bottom-up programming, where you write programs in multiple layers, the lower ones acting as programming languages for those above. If you do this right, you only have to keep the topmost layer in your head.

You mean people don’t do this?

In my mind this is the first Design Pattern; the most important one from which all other Design Patterns derive from and pay homage to. It was the most important thing I learned in college when studying networking models. It was the key to my understanding when I started writing software for the Macintosh under System 1. (Yes, I’m that old.) It helped me crack object oriented design. This design pattern: that you develop software in layers, which each layer supporting the one on top of it–this pattern is so ingrained in my mind that I cannot think of writing software any other way.

You can see a beautiful example of this approach in the TCP/IP networking stack: each layer of that stack from the hardware to the application layer is designed to do a very simple and predictable job, and to support the layer on top. The magic of sending a packet of data reliably across the world in an error-prone environment works because of a layer of components, where each component is designed to exactly one thing and do that one thing well–and each component relies upon the services provided by the service below to do that thing well.

My first parser was built this way: I wrote a layer of software that did nothing but tokenization, on top of a layer that did nothing but identify comments and stripped them out of the input stream, on top of a layer that did nothing but read bytes and track the current line number. And those layers sit on top of an operating system call that itself is built in layers: from a layer that does nothing but read blocks from the disk and spools them out in chunks at the request of the application to a layer which does nothing but communicate with the hard disk via a simple protocol, to a layer in the hard disk responsible for stepping the read head motor and interpret the magnetic pulses on the platter.

And on top of this tokenization layer was another layer which did nothing but read in and interpret the tokens representing the top level constructs of the language, which then called subroutines which did nothing but handle the language constructs for small pieces of the language.

I wrote (and have on my web site) an ASN.1 BER parser; it does nothing but handle reading and writing BER encoded byte streams. A man (whose name escapes me now because I’m at work–me culpa) has been in contact with me about writing an ASN.1 parser which uses my BER parser to handle tokenization; I’ve advised him to write code that handles the parsing, validation and interpretation of an ASN.1 specification. And in some cases I’ve said that essentially the BER parser layer isn’t the right place to handle certain features in an ASN.1 parser, such as validation of the input data stream.

See, part of the art of programming is also understanding what belongs in a layer, and what belongs in a new layer. Just as much as the art of programming is about understanding what pieces are part of the same functionality–and should therefore be part of the same layer.

At its root, each layer should be simple, self-contained, and do exactly one thing and do that one thing well. So if the code you’re working on doesn’t do this, then get up from your computer, go for a walk, and think about how to break it up into the proper layers.