Designing against decay

Having worked inside systems where a modest improvement requires hours of process, I've become cynical and eventually interested in how this happens. An institution can become better at following its procedures while becoming worse at the thing those procedures were meant to assist in improving. Why, given swaths of time, does such an arrangement that began to increase cohesive coordination become so difficult to administer and untangle?

A useful starting point is to separate an institution's purpose from the incentives of the people within it. Its existence depend on the continuation of its existence, its survival. Over time, in my experience, preserving its existence can become a more immediate concern than improving its decisional quality and outcomes ^[ 1 ]. On the surface this, this appears as a tug of war between short and long term constraints, though it's more interesting than that.

Suppose a slice of process provides one team control over a decision, while imposing a small delay on everyone else. Those people in question have a concentrated interest in retaining that relevance, a captured slice of territory. Scaled to an institutional size, the costs of which are dispersed among the rest of us who have to deal with the cumulative effects, suspended in a kind of institutional maelstrom of procedure. We may have neither the time nor wield the requisite "process capability" to get anything done quickly, or at all. Decisions that begin rational and reasonable can produce a dynamical whole that nobody intended be part of the original design.

Technology in general often carries many assumptions forward to simplify the interface of our environments for complicated coordination. Paper deeds become PDFs; wax seals become e-signatures ^[ a ]. Analogue processes become faster and more reliable, which is useful. However the hidden dance of state occurring behind the curtain have become increasingly difficult to deal with, further ossifying opaque decisional awareness to an army of specialist algorithms and administrators trying as best they can to deal with greater and greater informational flows. Advancement in this arena depends on mastering information through specialised process; and those in its orbit have good reason to become better at it, of course, even when those rules no longer serve a systems intended receivers.

Imperial China's civil service exams, the keju, provide an interesting historical example. They selected officials through demanding study of classical texts that helped sustain a proliferating administrative culture ^[ 2 ]. So effective, it shaped what ambitious people chose to learn. Societal status at the time depended on mastering a very particular curriculum devoted to such texts. This became so illusory, that many of the most capable minds were pulled to invest decades of their lives in serving administrative continuity, a synthetic and myopic version of reality.

These distortions can support continuity and constrain adaptation in varying amounts depending on the prescription and fanaticism of those involved. An institution may select exceptionally able people and channel abilities towards reproducing its existing false assumptions. Public sector efforts tend to suffer greatly from this type of decay partly due to the lack of a existential predator that feeds on lazy decisional quality. Public markets have PE to blame, though that's a hugely complicated problem in itself. Think of these distortions as a Keju amplification effect on an incorrect version of reality, very effective at propagating an untruthy latticework of process.

Specialised regulation, legal boilerplate, financial instruments and administrative tussles come to mind. That's not to say rules aren't necessary, they are, where a power station or an airport is a difficult to build, operate and maintain without a obsessive focus on good rule-making. Uncertainty here have a much closer binding with physical reality than the majority of rule contortions, perhaps born by a fundamentally more rigorous approach to risk.

"Process people" and resources can quite easily bury real intent with billable hours, a kind of informational camouflage. Complexity becomes a raw source of power when the cost and time requirement of decision-making exceeds what most people can afford. The Horizon scandal illustrates how serious the consequences can become when an institution successfully resists challenges to its own account ^[ 4 ] through process camouflage. These abusers tend to hide in plain sight, which the spiralling downward of decisions persist for many years before a headline brings all to bear, sparking sudden outrage and disgust.

The costs of these dynamics tend to be noticed most in physical infrastructure projects because they're physically observable. HS2 and Hinkley Point C have both faced substantial delays and cost increases ^[ 5 ]. Their difficulties have complex interrelations, certainly. They nevertheless unearth a practical question of observation: why do they take so long, what informational choke point exists that's adding years to decision-making capability, what billable hours are profiting most from added measurement, delay and more "unknown unknowns"?

In 2024, the median home cost 7.7 times the median annual earnings of a full-time employee ^[ 6 ]. This doesn't establish that process complexity caused the affordability problem in housing. It does, however, indicate a smoke signal demanding institutional attention whose decisions affect housing: can they improve the conditions under which people obtain a home? And I'm not suggesting a simplistic Help to Buy solution, there's reasonable evidence to suggest it helps corporate home builders sell more homes at inflated market prices, far more than it helps homebuyers. What would the actual solution be?

Public bodies and private firms both make extraordinary forms of coordination possible. The issue is how their incentives and outcomes change as these entities survive over time; when decisions cease to serve those who depend on them functioning at a net positive to outcome continuously, and provably. Those who exercise procedural sophistication at the expense of others who cannot impede spiralling effectors, with possible disastrous consequences, need to be removed from the equation that they no longer serve, as demonstrated by observed behaviour. There can't be an incentive to act badly and be rewarded for it on successive interactions, which unfortunately is the status quo in governance today.

We've responded to these frustrations in the past, most notably, Magna Carta placed constraints on royal authority and the American Constitution distributed power among institutions and the free market. Both enforced the rule of law as the fundamental channel by which things happen or don't happen in our civilised world. These were solutions to a huge problem of cohesive coordination. Their relevance here is the designs themselves, that is, it's their durable cooperation over time that remain effective, as measured by those faced with the consequences.

I like to think of good coordination, and legalese, as a machine for making fair decisions, provided we are very precise about what that means. Is a good decision also a fair one? Can an unfair decision also be good? How do we determine the difference? How do we prove a quality decision was made, and that it was fairer because it was higher quality?

There's a feedback problem here. A decision produces consequences; those consequences provide evidence about the decision - its collisions with reality. To learn, that evidence must be received by someone with the capability to act on its feedback, whatever that means. Our weighting on feedback back into important decisions we rely on will demand precision and scrutiny on decision quality which is going to force us to have to deal with these questions.

History seems to suggest that with each passing 100 years, decisions tend to care more about fairness than it did the previous 100 years. Therefore receiving feedback from anyone on the receiving end of being "decided on" is more important than it ever has been, yet it simply isn't respected and we continue to put up with low quality, opaque outcomes.

At Grenfell, repeated complaints about fire safety didn't instruct action ^[ 7 ]. On the list of events probable for a building, fire is persistently at the top. The translation of official complaints without the requisite actions and verification is a terrible way of dealing with reality, the sad truth being that, had Grenfell not caught fire, the building's Tenant Management Organisation would likely have continued to be judged as successful and fire-safe. How can feedback like this not be respected? The latent reality in most scenarios doesn't get measured or dealt with.

Good decisions and actions need to be inspectable and enduring until evidenced against. People should be able to establish why a thing exists, who benefits from it, who bears its costs and who can change it. Decisions need an identifiable owner and a record of the evidence used, based in reality, and I'm not referring to a normal distribution. Where risks are uncertain, those uncertainties should remain visible and we can all decide together if it's quantifiable and acceptable, as those who bear the consequences.

This does not imply that everyone must understand every technical detail. That would defeat much of the value of specialisation, and specialisation is needed in the world we live in. It does mean that someone affected by a decision should be able to question its outcome without having to acquire specialist procedural might in order to provide acted-on feedback.

The task then is largely an unglamorous one rooted in engineering, as are most things. Indistinct decisions made today must reorient on diverse participation by those that bear the consequences if they're to not decay and objectively considered as safe and sustaining. Our ability to detect and respond to embedded unaccountability requires equal sophisticated rigour. Process mass has unfortunately coaxed us towards a mirage of falseness. A good fairness machine is one that accepts evidence of failure indiscriminately to reliably help us correct what happens next, and provably so - by the skin of our teeth!

Footnotes