C5.5 Calibration

PMI library [12]; BIS backtesting [5]

The fifth component requires that estimates be confronted with what actually happened and that scales and models be recalibrated with the evidence: the likelihood bands of L3 against the event counts in the base, the detection ratings against the events that escaped detection, the simulated distributions against the losses that occurred. The principle comes from the project-management treatment of probability, which found that words like “possible” meant one chance in four to one person and one in two to the next, and insisted that estimates be checked against outcomes [12]; the discipline comes from the market-risk backtesting regime, in which a model keeps its standing only while the count of exceptions stays in the green zone [5].

What this reconciles is the forecast and the measurement that chapter 11 separated in the second cluster. An expected exposure is a forecast; a measured exposure is what happened; the gap between them is the only honest test of whether the controls worked, and the component is where the gap is computed and acted on. It applies to more than models. A register that rated a dosing error as rare, meaning less than once a year, and then recorded two in two months has a calibration problem before it has a control problem, and the component requires that the band be moved and the move be versioned, so that a trend across the change can still be read.