When does an AI alert become a fact?

Preserving the difference between a plausible explanation, an established finding and a decision that must be made now

Karina Carlos

A record that an AI system raised an alert establishes something about the system’s output. Whether the suspected event occurred remains a separate question. That distinction can disappear when an explanation sounds convincing, a customer needs an answer and someone has to decide what happens next.

The practical challenge is to act on the evidence available without making it carry more certainty than it can support. A decision may be necessary before the underlying question is resolved. Its justification should make that condition visible.

The payment and the unfinished finding

In my fictional story The payment is on hold and payroll is due, a customer says a held transfer funds tomorrow’s payroll. Mara revises wording that prematurely treats an AI alert as a finding of fraud. A documented provider change offers a possible explanation, but the instruction remains unverified. Lena, the duty fraud lead, takes the review; the payment stays on hold. Whether it clears before cutoff remains unknown.

The scene leaves the substantive outcome open. That is useful for examining judgment: we have to evaluate the quality of the reasoning before knowing whether the feared event was real. A correct eventual outcome would not, by itself, establish that every earlier inference was warranted.

What explanation research can and cannot establish

In CHI 2021, Gagan Bansal and colleagues studied sentiment judgments and logical reasoning. Explanations increased acceptance of AI advice, including incorrect advice, without significantly improving team accuracy over a confidence-display baseline. These were bounded tasks with crowdworkers, rather than specialist fraud investigations.

Zana Buçinca and colleagues studied 199 participants making nutrition choices with simulated AI. Cognitive-forcing designs reduced mistaken reliance when identifying the high-carbohydrate ingredient, compared with simple explainable-AI designs. The most effective designs received less favorable subjective ratings. The result concerns that experiment, not a universal benefit from adding friction.

Neither study establishes how to resolve a payment alert. My application to the fictional case is an inference: the design should help a reviewer examine the basis for a conclusion, and its evaluation should measure more than agreement with the system. A reassuring explanation is an insufficient success criterion.

Watch how the claim changes

Consider a hypothetical sequence of internal messages. The first says a transfer was flagged. The next describes suspected fraud. A later summary refers to a fraudulent transfer. If no additional evidence entered the process, the claim became stronger through repetition and wording alone.

This is a failure to preserve the status of a claim. It can happen even when everyone accurately copies the most recent summary. The next reader receives the conclusion without the uncertainty that originally qualified it.

I would ask a team to trace a consequential statement back to its source. What observation supports it? Which part is an interpretation? Where was it independently checked? If the only supporting material is another account of the same model output, the team should not count that repetition as corroboration.

Give uncertainty a useful form

Calling everything uncertain leaves the reviewer with too little to work with. A useful record distinguishes a verified observation, a statement supplied by someone else and an unresolved inference. Each can matter to the decision, but for different reasons.

An explanation of why a model produced an alert may help identify which inputs deserve attention. It does not automatically establish that those inputs are current, correctly attributed or sufficient for the conclusion being considered. Those questions require their own evidence.

In Human Authority, contextual discernment concerns the facts, criteria and consequences material to the present decision. I would apply that distinction here by asking what evidence could change the proposed course. The answer gives the investigation a direction and makes it possible to recognize when additional explanation adds no decision-relevant information.

Match the decision to the evidence

There may be sufficient grounds to request urgent specialist review while the grounds for declaring fraud remain absent. The evidentiary basis for a precaution and the basis for an accusation should be stated separately. Which actions are permissible depends on the institution’s controls, obligations and authorized decision-maker.

The same discipline applies to urgency. A claimed deadline may justify accelerating review without proving that the transaction is legitimate. Conversely, keeping a case unresolved can carry consequences that must be considered alongside the risk of acting. Neither pressure nor a wish to avoid responsibility supplies missing evidence.

For a decision under uncertainty, I would record the present basis, the unresolved condition and the event that requires reconsideration. This is an editorial proposal for making reasoning reviewable. It is not a payment-handling procedure or a substitute for applicable controls and specialist judgment.

Design a pause that contains actual work

An extra confirmation screen can still invite an automatic click. If a workflow asks people to reconsider a consequential recommendation, it should make clear what they are expected to evaluate and provide the means to do so.

One design worth testing would ask reviewers to identify a material missing fact and explain how it affects the current decision. Another could display the source and verification status alongside a consequential claim, making it harder for a model output to inherit the status of a checked fact. These are candidates for local evaluation, not established prescriptions for finance.

Evaluation should include mistaken agreement, mistaken disagreement and the burden introduced by the extra step. It should also examine whether the design is usable with the time and support actually available. A longer process is defensible only through the work it enables and the consequences it helps manage.

Keep the unresolved question visible

A handoff should preserve why the case is open. If the next decision-maker sees only a confident conclusion, the opportunity to question its basis may already have narrowed. If they receive an unstructured collection of caveats, the decision may be equally difficult to locate.

An alert becomes a basis for an established finding through verification appropriate to the claim and context. Until then, responsible judgment may still require action. Human Authority in that moment means making a defensible decision while keeping the limits of its justification available for challenge and correction.