Designing against decay

Having worked inside systems where a modest change requires hours of process, I've become fairly cynical about the process itself. An organisation can remain very busy long after it has stopped doing what it was set up to do. Over time, protecting its procedures and positions becomes more rewarding than fulfilling its purpose ^[1]. Each step has someone to defend it, even when nobody can explain what the whole arrangement achieves.

Technology often leaves this arrangement intact. Paper deeds become PDFs; wax seals become e-signatures. The document moves faster, but the assumptions about who must approve it, and why, remain largely unexamined. We reproduce the old process in software and consider the job done. Anyone whose role depends on an unnecessary step has little reason to ask for its removal.

Imperial China's civil service examinations, the keju, offer an older example. They selected officials through a demanding study of classical texts and helped sustain a common administrative culture ^[2]. They also gave ambitious people a strong reason to master the existing order. That is useful if continuity is the objective. It becomes a constraint when circumstances require a different kind of knowledge or judgement. The examinations alone cannot explain China's later economic divergence from Europe ^[3], but they illustrate how a system's method of selecting capable people can also limit what those people are encouraged to do.

The same incentive is familiar in modern institutions. Legal boilerplate, financial instruments and overlapping rules require specialists to interpret them. Some of that difficulty belongs to the problem. Some accumulates because each department adds its own requirements without taking responsibility for the result. Eventually, understanding the process becomes a profession in itself. Those who can afford the advice gain an advantage; those who cannot struggle even to establish what has happened. The Post Office Horizon scandal is a particularly severe example of how difficult it can become to challenge an institution's account ^[4].

The cost extends beyond the fees. Work waits for approval, decisions pass between departments, and each participant can point to a completed procedure while the original problem remains unresolved. Responsibility is divided so thoroughly that finding someone able to correct a mistake becomes another job.

British infrastructure makes the consequences physically very difficult to ignore. HS2 and Hinkley Point C have both suffered substantial cost increases and delays ^[5]. These projects have different problems, and procedural complexity cannot explain every overrun. But an organisation's ability to account for expenditure is a poor substitute for its ability to deliver a finished railway or power station. Housing presents a related test of what our institutions make possible: in England in 2024, the median home cost 7.7 times the median annual earnings of a full-time employee ^[6]. These are the outcomes by which people judge whether the arrangements governing their lives are working.

I don't think this is a useful argument against either public bodies or private firms. Both can do work that individuals could never coordinate alone, and both can become absorbed in preserving themselves. The question is what happens when they fail. Who bears the cost? Who has the information and authority to change course? If failure leaves profound levers responsible comfortably in place, there's little reason to expect they won't be used again.

Magna Carta placed limits on royal authority. The American Constitution distributed power among institutions whose competing interests could check one another. In each case, the rules changed the game that people in power were playing. The design problem is to make self-interest sustain open cooperation over time. Reputation, access to shared resources and the prospect of future cooperation can reinforce one another, so that sharing information and keeping commitments improve a participant's position in later rounds. The rules need to make that longer view worthwhile, even as people and circumstances change. Better governance is an upgrade to our capacity to coordinate as a species, across larger groups and longer periods.

Those rules lose their force when challenging a decision requires more time, money and specialist knowledge than most people possess. A formal right of appeal means little if exercising it is unaffordable. The institution can then treat compliance with its own procedures as sufficient evidence that it acted properly. A system that works this way will favour those who know how to use it, however impartial its rules appear on paper. Even perfect adherence to a procedure can reproduce a dangerous assumption. Its validity has to be tested against what happens in the world ^[7].

As the world becomes more complex, the systems governing it need greater bandwidth for information about what their decisions actually do. More algorithms are making decisions that affect more people, through processes whose full operation may be beyond any individual's understanding ^[8]. An explanation of every step may be impractical or impossible. Someone who has been decided on should still be able to report that the outcome is wrong, even if they cannot identify the faulty assumption or calculation.

That information has to reach whatever can change the decision. A complaint that receives a reference number and disappears has supplied no useful feedback. The system needs to retain the disputed outcome and the evidence, compare it with similar cases, and investigate whether its assumptions hold. A single justified complaint should be enough to correct an individual decision. Recurring dissatisfaction among people subject to the same rule should trigger a review of the rule itself. Where the evidence supports a change, that learning must alter future decisions, with a response to the people who raised it and a check that the correction worked.

The purpose of a system is what it does. If its decisions scale faster than its ability to recognise their consequences, its failures will scale too. The task is dull: engineering better dynamical systems that can detect and correct their own decay. An immune system recognises threats, responds to them and retains a memory that improves its response next time. Governance needs an equivalent capacity to detect manipulation and failing assumptions, and to adapt before the damage becomes entrenched. That defence must remain receptive to the people challenging the system. The capacity to learn from consequences has to grow with the capacity to impose them.

Footnotes