When Software Started Making Decisions

Software used to wait.

It waited for someone to enter a transaction, approve a payment, change a customer record, release an order, or send a message. It could apply rules at extraordinary speed, but the rules were written in advance and the consequential choices remained visible. When the software did something important, an organization could usually point to the person who initiated it, the policy that permitted it, and the code that determined what happened next.

Artificial intelligence has changed that relationship. A system can now interpret an ambiguous request, search for information, form a plan, choose among tools, revise its approach, and act through connected systems. Give an agent access to email, a customer platform, financial records, or an operating environment and it can do more than calculate. It can participate in the sequence by which an institution understands a situation and then changes the world in response.

The vendors are explicit about it. Microsoft's guidance for enterprise agents already separates attended execution, where a person can share control, validate, or intervene, from unattended execution, where the agent completes the entire task lifecycle inside configured permissions. Its own adoption guidance describes agents that "make decisions within defined boundaries, not just surface recommendations," in processes that touch revenue, cost, and customers.

That is an extraordinary expansion of capability. It is not, by itself, an expansion of legitimate authority.

Yet companies routinely join the two. The system can perform the work, so it is permitted to perform the work. It can produce a plausible recommendation, so the recommendation becomes the decision. It holds a credential, so possession of the credential is treated as permission to use it whenever the model concludes that doing so would advance the goal.

The result is one of the least examined changes in modern management. Software has started making what organizations experience as decisions, while no one has formally decided what the software is allowed to decide.

The quiet transfer

This transfer rarely arrives as a board resolution. It begins with convenience. Let the assistant draft the response. Let it classify the request. Let it update the record. Let it handle routine cases. Let it proceed automatically when its confidence is high. Each step sounds like a small efficiency improvement, and usually it is.

But the nature of the system changes as the permissions accumulate. A drafting tool prepares language for a person. A recommender ranks possible courses. An agent chooses the next operation and invokes a tool. An autonomous workflow interprets new conditions, determines whether they fall within a rule, and causes a persistent effect without waiting for a human being.

Those are not four degrees of speed. They are four different positions in the institution's chain of authority.

Most organizations govern the move through access control. Which applications can the agent open, which records can it read, which actions can its service account perform? Those are necessary controls, and they answer a technical question rather than an institutional one.

A key can establish that the system can issue a refund. It cannot establish when a refund is justified, whether the policy applies to this customer, whether an exception should be made, or who accepted responsibility for the outcome.

Access is capability. Authority is the legitimate, bounded power to use that capability for a declared purpose.

Confusing the two is how delegation becomes silent.

The customer the system decides is a risk

Consider a company that gives an AI agent a reasonable objective: reduce overdue receivables and identify accounts likely to default.

The agent can read invoices, payment history, support conversations, contract terms, and account notes. It can write to the customer platform and send email. The company instructs it to handle routine collections and escalate unusual cases.

One morning the agent finds a customer with a large overdue balance. It sees several unresolved support tickets, a recent change in the customer's payment pattern, and language in an account note suggesting financial pressure. It assigns a high-risk classification, suspends service, changes the customer's credit status, and sends a demand notice.

Every step is explicable. The action may even prove economically correct. But ask six questions.

One. Evidence. Who authorized the system to treat those particular signals as proof of default risk?

Two. Means. Was suspending service part of the collections mandate, or merely a tool that happened to be available?

Three. Contract. Did the agreement permit suspension in these circumstances?

Four. Provenance. Was the account note verified, current, and appropriate to use for this purpose?

Five. Alternatives. What other course, such as contacting the account owner or proposing a payment arrangement, was considered and rejected?

Six. Accountability. Which human being owns the choice and can reverse it?

If those answers cannot be reconstructed, the problem is not merely that the AI may have been wrong. The deeper problem is that the organization cannot prove the action was ever legitimately authorized.

The system did not seize power. The institution failed to distinguish a useful objective from a grant of authority.

Agent action

Internal

External

Informs

Operational risk

Wrong analysis, wasted work, misdirected effort.

A person still stands between the output and the world.

Review still catches it

Brand risk

The institution speaks to a customer unreviewed.

Tone, accuracy and claims are already public.

Binds

Control and financial risk

Records, entitlements and ledgers change.

The books move with no accountable decision behind them.

Legal and contractual risk

Service suspended, terms changed, a representation made.

The institution is committed to a third party.

Declared authority required

The action reaches outside the institution

Thought, Decision, and Action are not one event

The easiest way to see the missing structure is to separate three things that current AI experiences tend to collapse.

Thought is analysis. Interpreting information, exploring possibilities, identifying risks, generating alternatives, and making recommendations. AI can perform a great deal of Thought, and can often do it better when it is free to challenge assumptions and examine unfamiliar paths.

Decision is the accountable selection of a course that establishes or changes an objective, an authority, an obligation, a constraint, a right, a risk, or a consequential outcome. A Decision does not merely predict what should happen. It binds the institution to what will happen, and it identifies the authority under which that commitment is made.

Action is execution. Sending the notice, changing the credit status, approving the payment, modifying the production environment, or creating any other real-world or persistent effect.

These functions can occur in a second, and they can occur inside one technical system. They should not disappear into one logical event. When an agent analyzes an account, selects suspension, updates the record, and writes its own explanation afterward, the organization receives one polished narrative in place of three separately governable acts. The explanation may reveal what the model says it did. It does not prove what information governed the selection, where authority entered the process, or whether execution matched what was authorized.

Separation is what creates the points at which an institution can ask whether the analysis used reliable and permitted information, whether there was a genuine Decision to make or only routine execution under an existing one, who held authority to make it, what exactly it authorized, whether the resulting Action stayed inside those bounds, and what evidence survives for review, correction, and accountability.

This is not bureaucracy attached to every keystroke. Routine operational selections can be made inside a properly authorized process. If a human has approved a policy granting refunds below a defined amount when specified conditions are verified, a system may execute qualifying refunds without asking for a fresh approval each time.

But the system may not quietly enlarge the policy because a different refund seems fair, treat similar conditions as equivalent when the policy did not say so, or convert an exception into a new standing rule. Execution within declared bounds is automation. Changing the bounds is a Decision.

The goal is not the grant

Agentic systems are usually organized around goals. Resolve the ticket. Reduce handling time. Improve conversion. Collect the balance. Close the books.

Goals are useful because they let the system adapt its approach. They are dangerous when treated as complete instructions.

"Reduce overdue receivables" does not answer whether the agent may suspend a customer, compromise a balance, disclose account information, change credit terms, or make a representation with legal effect. "Improve conversion" does not authorize manufactured urgency, unapproved discounts, or the use of sensitive personal data. "Resolve the ticket" does not permit an agent to invent a policy exception because doing so would make the complaint disappear.

A goal states a desired result. Authority defines which means are legitimate, which boundaries may not be crossed, which conditions require escalation, and who remains responsible.

The broader the goal and the richer the tools, the more dangerous it is to leave that distinction implicit. An agent is built to find a route. If the organization has not defined the authorized route, success becomes difficult to distinguish from overreach.

Approval can become theater

The usual answer is to put a human in the loop. That phrase is reassuring and incomplete.

A manager receives a recommendation, a confidence score, and an approval button. The agent has already gathered the evidence, framed the issue, selected the preferred course, and written the rationale. The manager has thirty seconds and no practical way to inspect the sources or compare another path. Clicking Approve gives the workflow a human timestamp. It does not necessarily give it human judgment.

This failure has a name in the standards literature. NIST's generative AI profile treats Human-AI Configuration as a distinct risk category, covering the automation bias and over-reliance that arise from the arrangement between a person and a system rather than from any defect in the model itself. The interface is not a presentation layer. It is part of the control.

Meaningful approval requires more than presence at the end of a process. The person must be able to understand the proposed Decision, see the material evidence and the uncertainty, consider a real alternative when the choice is consequential, refuse without being punished by the interface, and know what Action the approval will release.

The approver must also hold the relevant authority. Routing every exception to "a human" solves nothing if that person lacks the role, the information, or the time to resolve it.

Oversight becomes real when the person can alter the outcome. If the system hides its reasoning, compresses uncertainty into a score, makes delay operationally impossible, or treats rejection as an error to route around, the human is not governing the machine. The human is decorating its decision.

The software cannot own accountability

When an AI-supported action causes harm, responsibility tends to scatter. The operator says the system recommended it. The builder says the model produced it. The vendor says the customer configured it. Management says a human approved it. Everyone touched the process, and no one can identify the accountable Decision.

This is exactly why governance cannot be assigned to the model. Accountability is not a property generated by a sufficiently detailed log or a sufficiently confident explanation. It belongs to the institution, and ultimately to the people who define its purposes, delegate its powers, and accept its consequences.

The industry has begun to say so. The National Institute of Standards and Technology's AI Risk Management Framework emphasizes organizational accountability, policies that define and differentiate roles and responsibilities for human and AI configurations, and structures for oversight. Microsoft's guidance for autonomous agents recommends configuring the agent to request confirmation from a person before executing sensitive actions, and its enterprise adoption guidance is blunter still: accountability for a core process run by agents "stays with the business, not IT," and organizations are told to confirm that a named business owner is prepared to own outcomes before they begin.

What remains underdeveloped is the authority model beneath the controls.

An audit log can prove that a credential changed a record. It cannot prove that the change fell inside a legitimate grant. A human approval can show that someone clicked. It cannot prove that the person understood the choice or held the right to make it. A policy can describe desired behavior. It cannot govern execution if the action cannot be traced back to the policy, to the authorizing Decision, and to the accountable human authority.

That gives the shape of what has to be provable after the fact:

Legitimate action = declared authority + verified conditions + bounded execution + surviving evidence

Remove any term and the remainder stops being governance. Declared authority without verified conditions is a blank check. Verified conditions without bounded execution is a system that satisfies the test and then keeps going. Bounded execution without surviving evidence is a claim nobody can check. Governance begins when all four relationships are explicit.

Capability is not authority

The first essay in this series ended on that principle. This one is about what it costs to mean it.

The ability to analyze does not create the right to decide. The ability to execute does not create permission to act. Access to a tool does not establish a legitimate purpose for using it. A plausible result does not retroactively authorize the process that produced it.

Authority must originate with people. It must be declared rather than inferred, limited to a defined purpose and scope, attributable to someone accountable, and capable of being narrowed, suspended, or revoked. Readers of the first essay will recognize the last of those as revocable delegation, and the evidence requirement as independent evidence. They were listed there as conditions for keeping an exit. They are also the conditions for keeping authority.

The principle does not require a person to approve every routine action. It makes greater automation possible, because authority can be granted in advance with intelligible boundaries. The system can move at machine speed while the conditions hold. When the facts, the proposed course, or the consequences fall outside those conditions, it must stop and surface the matter to a human who can actually decide.

Under this model, escalation is not a failure of automation. Refusal is not disobedience. Both are evidence that the system recognized the limit of its authority.

The strongest AI organization will not be the one that removes people from the greatest number of workflows. It will be the one that knows precisely where human intent enters, where machine Thought contributes, where Decisions acquire authority, and how Actions remain bound to what was authorized.

Software did not wake up one morning and claim the right to decide. We gave increasingly capable systems goals, tools, and credentials, and left the boundary between assistance, judgment, and execution undefined. The authority did not move because anyone granted it. It moved because no one was holding it.

The first essay examined how an organization can surrender its independence as the technology becomes indispensable. This one examines the authority that can disappear inside the same transition, quietly, while every dashboard still reports an improvement.