Aurorisk FR

Chapter 1 · Lesson 1.1

Why AI systems need their own analysis

A classic risk analysis assumes a system that always does the same thing, described by its specifications. An AI system learns from data, makes unpredictable mistakes and can change without anyone touching it: the analysis must allow for this.

It errs without warning

Classic software crashes or returns an error. A model always answers, confidently, even when it is wrong.

It inherits its data

Biases, gaps and errors in the training data show up in the results, though nobody wrote them.

It drifts

Real-world data changes, the provider updates the model: last month’s behaviour is not today’s.

It is opaque

It is not always possible to explain why an answer is given: challenging a decision becomes difficult.

It can be misused

A booby-trapped input (“prompt injection”), use outside the intended scope, excessive trust: attack or misuse is part of the risk.

It affects people

The harm often falls on third parties (candidates, patients, customers), not only on the organisation.

What this course adds

The course “Risks and AI” (coursia) presents the overview of risks and regulations. Here, we learn the method : from inventory to treatment plan and monitoring, on a complete case.

One analysis, three perspectives

The risk to people (rights, health, safety), the risk to theorganisation (legal, financial, reputation) and the security risk of the system itself (attacks on the model and the data). The three add up.

Key takeaways

An AI system errs with confidence, inherits its data, drifts, stays opaque and can be misused: a classic analysis does not see this. Risk is judged for people, for the organisation and for the security of the system.

Sources: Regulation (EU) 2024/1689 (Art. 9: risk management system); ISO/IEC 23894:2023 (guidance on AI risk management); NIST AI RMF 1.0.