“The first requisite for a body of practical and progressing thought: recognition of the kind of problem at issue. Lacking this, it has found the shortest distance to a dead end.”

Jane Jacobs


Why do recurring problems not get solved? Why do good intentions produce disastrous outcomes? One significant and underappreciated element is the importance of problem solving methodology. The issue Jane Jacobs is describing in this quote is really one of form. This is what she means when she talks about identifying the ‘kind’ of problem. The necessity of understanding the ‘kind’ of form a problem takes is evidenced in her next line; the ‘shortest distance to a dead end’ – the extent of what can result from making this mistake. Splitting this warning we find two important ideas. The first is futility, the ‘dead end’ represents defeat through the image of finality. The wrong solution will quite literally take you nowhere. At first the ‘shortest distance’ comment appears to be humorously condemning, the misguided ‘solver’ furiously reenacting their failures, like a bird knocking itself against the open window. There is more to be uncovered here, however. Through the repetition of this mismatched solution, the players start playing a different game – who can find the ‘shortest distance’. To put it another way, we can think of the bird’s problem as ‘lock x’, but the bird thinks that only ‘key y’ will fit this lock, and the bird repeatedly flies into the window to prove this point. Here we will consider why this mismatch between lock and key occurs on both an individual and institutional level, and how we might identify when it is happening. 

There are any number of reasons why someone might attempt to use ‘key y’ on ‘lock x’. It might be the only key they have, or trained to use, or their boss told them to use ‘key y’ or organisational incentives may lead them to use ‘key y.’ There are also other groups of narratives, usually focused on individual failings, from incompetence, thoughtlessness, laziness, to lack of imagination and greed. The two groups of reasons, organisational and individual failing, and their relationship reveals a truth; that whenever there is a mismatch between problems and solutions which are repeated, it is rarely the result of individual acting alone. Addressing systemic problems by blaming individual actions can never succeed. Worse, it distracts. By exerting energy on the individual we risk losing sight of ‘lock x’, and which key is ‘key x’, our real goal.

To use a practical example, take sports doping. If we take the prevalence of this to be a result of an individual’s actions then we are also saying that some people, or even most people (Phillip Hurst, in his article “The role of moral identity and regret on cheating in sport” found 70% of competitors had cheated) are fundamentally and uncritically bad. Or, we can recognize that the repetition of unwanted actions and outcomes is a hint that the system is misaligned, that there is something at play that causes these people to believe ‘key y’ will fit this time, and that the process of trying has no adverse consequences. Again, we get wrapped up in narratives and bind ourselves in binary, unhelpful thinking. Think how easy it is for an individual to self-rationalize bad behaviour when they lose sight of the goal (sporting excellence) and focus only on money and prestige; how easy it is for them to believe that, “only bad people cheat, and I am not a bad person, therefore I cannot cheat”, before administering PEDs. At the same time this mindset affects sports organisations. For sports administrators, journalists and sponsors it is easier to focus on the ‘bad apple’, than the issues that lead to the systematic use of the wrong key. The rate of institutional failures we currently experience shows that we haven’t solved this problem of repeatedly using the wrong key to open the right lock. 

The issue of misaligned key and lock forms is critical at the institutional level because of scale of impact and the difficulty of changing organisational behaviour. When choosing ‘key y’ organisations argue practicality, simplicity, cost, time, what is measurable, tradition, obligation as potential justifications. Ultimately, the linking factor between all of these ‘reasons’ for failure is a combination of inertia, lack of curiosity and organisational blind spots. Each new problem is a new lock, and it will require a new key. As Alinksy says, ‘no situation ever repeats itself, [so] no tactic can be precisely the same’. In other words, an inability to start again, to think at the systems level, to be able to see the ‘shape’ of the problem and the form of the solution will inevitably lead to failure. The practical risks of misapplying a solution is brilliantly articulated in Andrew Gordon’s book ‘The Rules of the Game: Jutland and British Naval Command’. Like Jacobs, Gordon knows that understanding the form of the ‘lock’ takes is the unskippable first step in solving a problem. So important is this step, it is enshrined by the title. You must know the ‘rules’, and the ‘rules’ have to be aligned to the problem if you are to have any hope of winning.

The British Navy and its attachment to signalling with flags against the German’s use of radio displays several of the features of a misaligned system. He describes the cultural inertia; describing those with ‘vested interests in promoting signalling culture … had been saying “We can do this. Trust us!”, and had resisted attempts to de-skill signalling or reduce its role in fleet-tactics’. Gordon also gives ‘dignity, reputations, career prospects and funding’ as other examples of ‘vested interests’. Of course, after the application of the wrong key follows the moral justification; the failure of flag signalling in battle conditions was ‘not their fault because the task was, after all, so difficult’, Gordon says with some scorn. Then there is the advancement of technology; radio permitted instantaneous, long-range communication between coal-powered ships that produced a thick smog. Strict adherence to the belief that flag signalling was the right solution for the problem of communication meant the Navy was ideologically unable to keep up with the times in the same way their enemies were. ‘The advent of new technology assists the discrediting of previous empirical doctrine’; a change in technology destroys the utility of previous tactics. Significantly, Gordon also makes it clear that in the Royal Navy there were multiple reinforcing organisational incentives in place which maintained custom and practice. He lists:

  • ‘”It is acknowledged that the highest attainment of a seaman is to manoeuvre a fleet in close order (and so you need look no further than my Manoeuvring Book for tactical doctrine).”
  • “It may be taken for granted that future battles will be fought on parallel lines and straight courses (and so my otherwise inadequate fire-control system answers the Fleet’s foreseeable needs).”
  • “Whoever shoots fastest will gain the ascendancy and keep it, as the other fellow can’t see to reply (and so the BCF’s lousy marksmanship isn’t really an issue).”
  • “It is a universal truth that an admiral should inform his subordinates by signal before charging off over the horizon (and so tactical legitimacy is bestowed by us signallers).”’

The institution formalised its inertia and created stories explaining why still was the only right thing. The real purpose of signalling – communication – was lost in this muddle, and lives too in the process.

It would be contradictory to offer a singular solution to the problem of ‘key y’, and so instead it may be useful to identify some clues we can look for that tell us we have found the ‘shortest route to the dead end’. Presented in the form of a non-exhaustive list: (a) Attachment to a closed mindset. Things that work, work. They do not require rationalisation (b) When there is repeated failure to meet stated outcomes. This can be a sign that there is a different, hidden goal cuckoo-ing the first. Take politics – do politicians fail to deliver on their promises because all of them are incompetent, stupid, lazy etc., or is it because the politicians true ‘lock’ is not satisfying their constituents, it is re-election. The only reason to implement policy is if it helps achieve that goal. The failure is a ‘feature, not a bug’. It is hard coded. (c) Be wary of the ‘bad apple excuse’. System problems are almost never caused by a single person. Beware of institutions that are quick to condemn individual behaviour as the source of all problems. If all your apples are rotting, there is probably something wrong with the tree. (d) Look for double-downers. Institutions that respond to failure by insisting that their solution is definitely the right one, even after demonstrated failure, are terminally static. They are hiding from the problem, unable or unwilling to accept their mistakes and try something different. This group is most visible by their declaration in failure – “this will work next time, if only we had more money!”.

Failure to understand the precise form of a problem results in disastrous consequences as it makes us blind to the actual requirements needed to construct the solution. We get stuck running in circles down Jacob’s ‘dead end’ road, throwing more and more of the same at a problem and delusionally expecting to get different results. If a problem is to be efficiently and purposefully solved, the problem solver must first stop and consider what exactly it is they are trying to do, and whether their proposed solution actually works towards solving that particular problem, or they might end up like Tracy and Jenna in 30 Rock.

Posted in

2 responses to “A Question of Form: system alignment in problem-solving”

  1. amcintyre13 Avatar
    amcintyre13

    Great post. I’m always a sucker for a ‘Rules of the Game’ reference. The interesting thing about the late 19th and early 20th century operational calcification of the Royal Navy was that it was selective. The Dreadnought and the pivot to fast battleships with large calibre uniform armament was truly revolutionary because Jacky Fisher framed the macro problem as ability to deliver maximum weight of broadside and pushed for the right key for that lock faster than any other navy. But rather than learning the Nelsonian lesson that you should work back from winning to rethink all strategies, tactics, and operating processes the Navy began to believe that the same approach would always produce the same results if continually replicated, resulting in the self-validating orthodoxies highlighted in the quotes. My own field of financial services is littered with examples of recurring problems not being solved because the current orthodoxies dominate thinking and don’t open the lock. It then takes a Paypal, Stripe or Vanguard to show what is possible and change the rules of the game, because they first define the lock to be opened and then cut their own keys.

    Liked by 1 person

  2. S Avatar
    S

    It is interesting to read this and then to reflect on what happened in the post office & horizon situation.

    Like

Leave a comment