Chapter 1 · Lesson 1.2
What AI can do, and what it cannot do
A language model has read many HAZOPs, accident reports and standards. It therefore writes HAZOP lines that look very professional. That does not mean it knows your installation.
It can
- List the parameter × guide word combinations without missing any through fatigue
- Propose classic causes and consequences for a type of equipment
- Rephrase a confusing line, harmonise the vocabulary
- Spot the inconsistencies in a table (cause without consequence, vague recommendation)
- Summarise operating experience or an incident report that you supply
It cannot
- Know the real installation: tags, sequences, instructions, equipment condition
- Guarantee that a barrier exists, works and is independent
- Estimate a severity without knowing the populations and the effect zones
- Remember what happened on the site
- Take responsibility for the study
Three behaviours to know about
- Overconfidence : an invented barrier is written in the same tone as a real one.
- Generality : without context, the causes are those of any reactor (“equipment failure”, “human error”).
- Compliance : if your question assumes an answer, the model tends to confirm it.
The trap
The risk is not that AI gets it wrong in a visible way, but that it produces a table that is apparently complete and that the team reads too quickly.
The mechanisms (hallucination, bias, drift) are explained in the course “Risks and AI” (chapter 5, “Using AI well”). Here we apply them to HAZOP.