Where LLMs actually help in functional safety — and where they must not

Two conversations happen around AI in safety engineering, and both are unproductive. One camp says language models have no place near a safety-critical program. The other pastes a HARA template into a chatbot and calls the output analysis. I use LLM-assisted workflows daily on a real safety program, so here is the working split I have arrived at: where the leverage is real, and where the line is hard.

The distinction that matters: acceleration versus judgment

Safety work has two kinds of content. There is judgment — deciding a hazard’s severity class, accepting a residual risk, signing that an argument holds. And there is transformation — restructuring, cross-referencing, reformatting, and drafting around judgments already made. The second category is most of the hours on a safety program, and it is where language models earn their keep. The first category is not delegable. Not because models are weak today, but because a safety case is a chain of accountable decisions, and an assessor asks who decided and on what evidence — a question that must have a human answer.

Where the leverage is real

First drafts of structured documents. A functional safety concept has a known shape. Given the safety goals, the architecture, and the operating modes, a model produces a structurally complete draft in minutes — every section present, every safety goal addressed, terminology consistent. I then do the actual engineering on a complete skeleton instead of a blank page. The review effort is unchanged; the drafting effort collapses.

Consistency checking across large requirement sets. Asking a model to sweep several hundred requirements for contradictions, orphaned references, undefined terms, and requirements that quietly restate each other finds real defects — the kind human reviewers miss because reading requirement 412 with requirement 87 still in mind is exactly what humans are bad at. Every finding gets verified by a person; the point is that the sweep is exhaustive and cheap.

Building the tools themselves. This is the least discussed and highest-value use. The requirements-management application and the FTTI simulation tool I use in my current work were both built with AI-assisted development — full-stack tooling shaped exactly to the safety workflow, delivered by one engineer in a timeframe that would previously have required a team. The safety case does not care that a tool was AI-assisted; it cares that the tool’s output is verifiable. Which it is — that is what the tool is for.

Where the line is hard

• No model output enters the safety case unreviewed. Everything an LLM touches is a draft until a qualified engineer has judged it. This is not ceremony; models produce confident, well-formatted text that is occasionally wrong in exactly the places confidence matters.
• No judgment calls. ASIL classification, risk acceptance, verification sign-off, confirmation measures — human, always, with rationale recorded.
• No fabricated evidence. A model asked to “write the test report” will write one. The failure mode is obvious and career-ending. Evidence is generated by executing things, never by describing them.
• Tool confidence still applies. If a workflow’s output can inject an error into a safety work product without detection, ISO 26262’s tool-classification logic applies to it regardless of how fashionable the tool is. The honest answer for LLM-in-the-loop workflows is usually to design the process so detection is guaranteed — human review as the mitigation — rather than arguing the error cannot occur.

The standards are catching up

ISO/PAS 8800 addresses safety of AI inside road vehicles — a different question from AI in the engineering process, but its discipline around data, validation, and monitoring previews how assessors will think. My expectation, watching this space closely: process-side AI use will be judged the way tool use has always been judged — by whether errors can propagate undetected. Teams that build the review structure now will not need to retrofit it when the question appears in an assessment.

The engineers who benefit are not the ones who resist the tools or the ones who trust them. They are the ones who can say, precisely, which parts of their work are transformation and which are judgment — and automate only the first.