On system reliability, or, a top-down approach to (dispel the myth of) "software engineering"

After establishing that "software engineering" can only be considered an honest engineering discipline through the practice of making software suck less, the next logical step is figuring out how to do that.

Software engineering methodologies are dime a dozen, and we might get to them sometime later, but for now let us start from the system-level view¹.

As a tradition² software engineers build their (general or narrow-purpose) systems using the "commodity component-based" philosophy, i.e. instead of rewriting software, they aim to reuse it as much as possible. This approach may look sensible on the surface, except when it isn't. In fact the mantra of "modularity" is nowadays flagrantly overused in the field, as engineers either ignore or simply do not know the trade-offs involved. The same is the case with software's so-called "flexibility", which supposedly makes software cheaper to build than hardware.

In this essay I will attempt to deconstruct this principle and its supposedly magical properties by enumerating a set of trade-offs (or problems, if you wish to call them such) that are particular to the approach. The first two trade-offs stem from purely human issues.

Trade-off 1: standards. Engineers generally use standards to guide the building of their systems. In software this is rarely the case³, with many so-called "engineers" not even bothering to specify their software, let alone standardize it. Thus very often a single implementation of a program becomes the de facto standard, even in the free and open-source software world. I'm not even touching closed-source software with a ten foot pole here, that can of worms is too rotten.

Trade-off 2: Abelson's saying. Good software is in some sense not that different from good literature, e.g. well-known scientific papers. The latter are often partially or completely rewritten by multiple people, sometimes (more rarely) using multiple languages, thus giving the writer as well as the public a better idea of what the work says and does. On the other hand single designs and implementations are harbingers of mono-cultures, which lead to the narrowing of ideas⁴ in the field. Of course, sometimes new ideas are worse than no ideas.

The next two trade-offs are pure design considerations.

Trade-off 3: dependencies. The existence of granular components gives rise to (sometimes circular) dependencies, e.g. component D needs component C which relies on A and B to work. The number of implicit relations in dependency trees (or graphs) tend to grow exponentially, which makes solving dependencies a (generally) hard problem⁵.

Trade-off 4: component versioning. Components' interfaces change in time, which is especially problematic in the case of components with lax (or no) interface specification. This may give rise to situations such as (but not limited to) that in which D needs C needs A1 and B, while E needs A2 to work, where A1 and A2 are different versions of A. It may be that A1 and A2 cannot coexist in the system -- this is an addendum to trade-off 3, and it often makes engineering "commodity component-based" systems intractable, especially when so-called "software upgrades" are involved.

Finally, the fifth trade-off is of a mixed nature:

Trade-off 5: fits-in-head. Systems with a large number of components usually do not "fit in head", i.e. they are not fully understood by the engineers involved in developing it. This is another generally hard problem, the chief idea being that the bigger the number of components, the bigger is the number of unknowns. With this the probability of non-deterministic behaviour in the system grows and with it the system's overall fragility.

These trade-offs are far from being the only issues of the "commodity component-based" approach and are not necessarily inherent to it. For example the first two trade-offs are observable in other approaches too, but bear more weight in the philosophical framework of code reuse.

As things usually go in any honest engineering discipline, one cannot expect to find a magical solution to all these trade-offs, otherwise they would be something other than trade-offs -- and the discipline would yet again be something other than engineering. However, we can derive a set of general principles to be used as rules of thumb, and broken when absolutely necessary, as expected of engineering. These principles can be formulated as follows:

Principle 1: clarity of purpose. Systems must be ideally designed with a single purpose in mind (or, to solve a single problem) and as a consequence must be as small as possible. The smaller its "number of purposes", the more resilient the system will be.

Principle 2: integrated structure. System designs must be monolithic. This does not preclude the use of components, as long as said components are fully integrated into the system⁶.

Principle 3: convergence. "Software upgrades" are to be avoided. Ideally the work will converge to a largely stable code base. Adding features or changing the system's scope should ideally result in a complete redesign and rewrite.

This list of rules of thumb is also necessarily incomplete, but we will keep it short for the sake of enumerating a few principles rather than rigid rules. More importantly, they may be taken as the first steps toward an honest, sane, enthusiasm-free, minimal-suck approach to software engineering.