The heat map lies

We have fourteen risks, each written as a proper sentence, each with a probability that has been asked what it is a probability of, and each with an impact anchored to something real.

The next step is the one every organisation takes. Put them on a grid. Probability up the side, impact along the bottom, red in one corner, green in the other. Colour it in, take a screenshot, put it in the monthly report.

So let's do that, and watch what happens to three of them on the way.

A pump that fails during a pour: $12,000. Steel escalation over the next two years: $30,000. Rock under forty-two piles: $60,000.

The company's scale says medium impact is anything from $10,000 to $75,000.

All three are medium.

A hire pump, a steel order and the ground under the entire substructure arrive at the board as the same word, in the same colour, on the same row. One of them is five times the other and the report has no way of saying so.

ONE BAND. FIVE TIMES THE MONEY. Company scale · Medium impact = $10,000 to $75,000 $12,000 Pump failure MEDIUM $30,000 Steel escalation MEDIUM $60,000 Rock in the pile bores MEDIUM Same word. Same colour. Same row in the report.
Figure 1 — One band, five times the money. A pump, a steel order and the ground under forty-two piles all arrive at the board as the same word.

A finer grid doesn't fix it

The obvious answer is more bands. Go from three to five, narrow them, and the collapse should go away.

It doesn't. It just moves.

Put all fourteen on a five-by-five and two things happen immediately. Eleven of the fourteen land in the middle band — the amber diagonal that runs from bottom-right to top-left. And nothing at all lands in the corners. No risk on this project scores 1 or 2, and none scores 20 or 25.

That isn't a property of this project. It is a property of the exercise. Nobody scores their own area as trivial, because then why is it on the register. And nobody scores it catastrophic either, because that means a meeting with the board and a conversation about whether the job should have been bid at all.

The grid has twenty-five cells and about six of them are ever used.

FOURTEEN RISKS ON A FIVE-BY-FIVE 1 2 3 4 5 2 4 6 8 10 3 6 9 12 15 4 8 12 16 20 5 10 15 20 25 1 2 2 2 1 4 2 IMPACT → PROBABILITY ↑ all score 12 Rock · Steel Late detailing 11 of 14 sit in the amber band together Nothing is scored 1 or 2. Nothing is scored 20 or 25.
Figure 2 — Fourteen risks on a five-by-five. A finer grid does not fix the collapse. Three risks land on the same score, and almost everything drifts into the middle band.

You cannot multiply a rank

Now the part that is a genuine mathematical problem rather than a behavioural one.

The bands are labelled 1 to 5. That looks like a number, so people multiply them. Probability 4 times impact 3 gives a score of 12, and the register gets sorted by that column.

But those aren't numbers. They're ranks. Band 4 is not twice band 2, in the same way that fourth place in a race is not twice as fast as second. The bands are just labels in an order, and the gaps between them are not equal — on this scale, the jump from band 2 to band 3 is $10,000, and the jump from band 4 to band 5 is unbounded.

Multiply two labels and the answer is a label with no units. It looks like a priority. It is a number with no meaning attached to it, and it will happily rank risks in the wrong order.

Watch it rank them wrongly

Three of our risks score exactly twelve.

Late detailing: 55% probability, $20,000 impact. Steel escalation: 50%, $30,000. Rock: 40%, $60,000.

On the grid all three are the same. Same score, same cell colour, same position in the sorted list. A project manager with time to deal with one of them has no basis for choosing.

Now do it in money. Late detailing is worth $11,000. Steel is worth $15,000. Rock is worth $24,000.

The rock is worth more than twice the late detailing, and the grid put them level. Worse — because the rock has the lowest probability of the three, it will usually appear below the other two in a list sorted by likelihood first, which is how most registers are actually read.

The grid didn't just fail to help. It pointed at the wrong risk.

SAME SCORE. DIFFERENT MONEY. SCORE EXPECTED VALUE Late detailing 55% × $20,000 12 $11,000 Steel escalation 50% × $30,000 12 $15,000 Rock in pile bores 40% × $60,000 12 $24,000 The grid ranks these three equal. The money says one is worth more than twice another — and it is the one you would fund last. 2.2×
Figure 3 — Same score, different money. Three risks tie at twelve. Their expected values run from $11,000 to $24,000, and nothing on the grid tells you which is which.

So why does everybody use one?

Because it is genuinely good at one thing, and that thing is worth having.

A heat map is a triage tool. Its job is to take forty items nobody has thought about and sort them into look at this now, look at this soon, and we have written it down and that is enough for the moment. For that, ranks are fine. You do not need to know that the rock is worth $24,000 to know it belongs in the first group.

What it cannot do is tell you how much money to hold. The moment somebody adds up a column of scores, or presents a total, or uses a colour to justify a contingency figure, the tool has been asked a question it was never built to answer.

A heat map is a queue, not a valuation.

And that is the honest boundary of qualitative analysis. It is quick, it needs no data, it can be done in a room in an hour, and it is completely sufficient for deciding what to look at first. It is never sufficient for deciding what to pay.

Practical insight

Take your own register and add one column next to the score. Probability times impact, in money, in full. No bands, no labels — the actual multiplication.

Then sort the register by score, print it, sort it by money, print it again, and lay the two lists side by side.

If the top five are the same on both, your bands happen to be well chosen for this project and you have lost nothing. If they aren't — and on most projects they aren't — then every decision anybody has made from that register was made from the wrong order.

It costs about ten minutes, and it is the single fastest way to find out whether your risk process is doing anything at all.

Key takeaways

✔ On a three-point scale, $12,000, $30,000 and $60,000 are all medium. Five times the money, one word.
✔ A finer grid moves the collapse rather than removing it — eleven of fourteen risks landed in the middle band.
✔ Nobody scores their own area trivial and nobody scores it catastrophic, so twenty-five cells become about six.
✔ Band numbers are ranks, not quantities. Band 4 is not twice band 2, and multiplying two ranks produces a label with no units.
✔ Three risks tied at a score of 12 while their expected values ran from $11,000 to $24,000.
✔ The lowest-probability risk of the three was the most valuable, and a register sorted by likelihood buries it.
✔ A heat map is a queue, not a valuation. It is enough to decide what to look at first, never enough to decide what to pay.

What's coming next

The grid is a queue. Fine. But a queue is only as good as the numbers that formed it, and every number we have used so far came out of somebody's head.

Two of five boreholes. Twenty-one piles. Forty percent. Every one of those was a person making a judgement, and people do not make judgements randomly. They make them wrong in a particular direction, over and over, in a way that is measurable and has a name.

Next week we look at why the estimate came in at $998,612 against a tender of $1,000,000 — a gap of less than a quarter of a percent — and why that is almost never what a calculation produces. It is what a target produces.

Enjoyed this lesson?

Join with Google to get each new lesson the moment it's published — and help me see which topics matter most to you. No spam, one email a week, unsubscribe anytime.