An answer can be useful without being a sufficient basis for action. A forecast might help a team explore its options without justifying a commitment to a customer. An invoice category might help organize work without establishing that a payment should be released.
In Article 01, “Permission Is Not Reliance,” I distinguished a system’s permission to act from an organization’s justification for relying on its output. There is a related distinction worth examining: confidence in a result and the decision to use it.
When software uses a result to initiate action, that distinction has practical consequences. The organization is entrusting something to the result. It needs to be clear about what that is.
Confidence answers part of the question
Confidence can help describe uncertainty about a result. It can inform a decision, but it does not by itself establish that the result is suitable for every use.
NIST’s AI Risk Management Framework 1.0 treats trustworthiness as broader than accuracy. Its characteristics include safety, security, accountability, privacy, and fairness, considered in the context of use. [1]
For leaders, the implication is straightforward: evidence that a system performs well matters. The purpose for which the organization intends to rely on it matters too.
A recommendation to consider a supplier and a commitment to purchase from that supplier carry different consequences. Confidence in the recommendation does not erase that difference.
A useful result can have a limited purpose
Consider a hypothetical demand forecast. It may be useful in a planning discussion. Relying on it to promise delivery to a customer involves a different business commitment.
Now suppose circumstances change after the forecast is produced. A supplier’s available capacity may no longer be what the planning team understood it to be. The forecast still exists, and the software that produced it may still be operating normally. That alone does not establish that the customer commitment remains justified.
This does not require a malicious agent or a broken model. The business use can become inappropriate even when the technology remains capable of doing its assigned work.
NIST’s framework recognizes that acceptable risk depends on the application and context, and that risk tolerance can change over time. [2] My point is about the organizational consequence: the case for relying on a result cannot be assumed to travel unchanged into every situation.
The question extends beyond AI
A rules-based application can calculate exactly what it was instructed to calculate and still provide an unsuitable basis for a different decision. An optimization tool can produce a useful comparison without resolving whether a particular commitment is appropriate.
The reliance question therefore belongs in discussions about AI and other computational systems. It applies across industries and does not depend on a particular vendor, platform, or method.
That breadth matters. An organization should not overlook a consequential use merely because the technology behind it is familiar.
What leaders should take from this
Automation discussions often begin with speed, cost, and performance. Those are legitimate measures of value. They become more meaningful when leaders also understand what decisions the organization is prepared to entrust to the resulting outputs.
I use Runtime Reliance Governance to name this business problem: whether a computational result should be relied upon for a particular consequential use. It is the problem I am working on through spotTED.
A useful question for an executive discussion is: what are we justified in doing with this result, in this situation?
Confidence in an answer deserves attention. So does the commitment made by acting on it.
References
[1] NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023, Section 3, printed page 12. https://doi.org/10.6028/NIST.AI.100-1
[2] NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), January 2023, Section 1.2.2, printed page 7. https://doi.org/10.6028/NIST.AI.100-1