Chapter 7 — Why rules broke down (the five walls)

Wall 1 — You can't articulate what you know

"We know more than we can tell." — Michael Polanyi

You recognise your friend's face instantly. Now write the rule. Nose-to-eye distance? Skin tone? You cannot do it — and neither could the domain experts being interviewed for expert systems. Their real skill was pattern recognition learned from thousands of cases, not a rule list.

Wall 2 — Brittleness

A rule system is exactly as smart as its rules and not one bit smarter. Give it something slightly outside what you anticipated and it fails — often silently or absurdly. It has no graceful degradation. Compare:

Rule system asked "How much annual leave?"    -> "Sorry, no answer."     (nothing)
ML system asked the same                       -> "25 days" with 78% confidence

The ML system generalises from similar training examples. The rule system can't generalise at all.

Wall 3 — Exceptions all the way down

IF bird THEN can_fly
   ... except penguins
   ... except ostriches
   ... except a bird with a broken wing
   ... except a dead bird
   ... except a bird in a cage
   ... except a baby bird
   ... (this list never terminates)

This is called the qualification problem. There's a sibling, the frame problem: after any action, you must also state everything that didn't change (painting a wall doesn't move the furniture). Formal logic needs an axiom for every non-effect. The real world has infinite non-effects.

Wall 4 — Combinatorial explosion

Rules interact. With 600 rules, the possible chains are astronomically many. Add rule 601 and you may silently break an interaction between rules 12 and 340. Rule bases become unmaintainable at exactly the size where they'd become useful.

Wall 5 — The symbol grounding problem

To a rule engine, CAT is just a token. It has no connection to any cat, image, sound, or experience. Symbol manipulation alone never touches the world. (This critique applies, in a modified form, to LLMs too — see the "Debated" note in Chapter 31.)

What happened next

Two AI winters (funding collapses: late 1970s, and again after the expert-system industry crashed around 1987) and then a decisive strategy change:

OLD:  interview experts  ->  write rules  ->  system applies rules
NEW:  collect examples   ->  machine finds patterns  ->  system predicts

      and accept that answers are PROBABLE, not CERTAIN

That acceptance — probabilistic instead of certain — is the doorway to Part 3.