How we establish that a prediction is worth acting on
A predictive safety system should be assessed on evidence, not on the confidence of its interface. These questions are agreed with you before any predictive output is relied upon operationally. They are normally settled in a single facilitated session with your HSE and IT leads, and they form the basis of the pilot.
What is being predicted
The predicted outcome must be specified precisely: the event type, the unit of prediction — asset, area or site — and the time window within which the prediction applies. A useful window is one that leaves sufficient time to intervene. "Risk is elevated" is not a prediction. "Elevated probability of a gas release at this manifold within 72 hours" is.
The base rate
How frequently does the predicted outcome actually occur? Serious incidents are rare events. A model that consistently predicts nothing will appear highly accurate and provide no value whatsoever. Understanding the base rate is what allows genuine performance to be distinguished from arithmetic artefact. This is the single most common way predictive safety accuracy figures mislead.
The acceptable error balance
False positives consume inspection and response capacity. False negatives are missed warnings. The two trade against one another, and the correct balance is a business decision for you rather than a technical one for us. A refinery in turnaround and a distributed tower portfolio will sit in different places, and both are correct.
The intervention question
If a warning prompts an inspection and no incident subsequently occurs, was the prediction incorrect, or was it successful? This has a material effect on measured accuracy and must be resolved before measurement begins. We recommend that interventions and their outcomes are recorded, so that prevented events are not counted as false alarms.
Reporting behaviour
In safety data specifically, reporting culture confounds analysis. An increase in near-miss reports may indicate rising risk, or improving reporting behaviour. These have opposite implications and look identical in the data. Where reporting rates vary materially between sites, shifts or supervisors, that variation must be understood before signal volume is treated as a risk indicator.