AI and the Accountability Lag
Scientific publishing has long depended on an imperfect assumption: producing a credible paper usually costs enough that its publication offers evidence that substantive work occurred. AI destabilizes that assumption. A fabricated study can reproduce the signals associated with genuine research—method, data, citations, disciplinary language—without reproducing the work behind them.
This divergence exemplifies what I call the accountability lag: the widening distance between the rate at which AI expands consequential action and the rate at which institutions can govern that action. The lag arises because technical capability and institutional responsibility follow different scaling patterns.
Bruce Schneier’s four dimensions of AI advantage help explain the technical side of that difference: speed, scale, scope, and sophistication. AI can complete an activity faster, repeat it across more settings, extend it into more domains, and incorporate more interacting factors. As those advantages compound, the activity surrounding the technology begins to change. Yet the institutions governing that activity do not necessarily undergo the same transformation. Operational capacity acquires the four S’s; responsibility remains tied to scarce resources such as attention, authority, expertise, evidence, and enforceable consequences.
The imbalance can be expressed in terms of marginal cost. Producing another AI-generated output or action may cost very little. Meaningful accountability generally requires attention to the particular case. Each disputed diagnosis, denied benefit, defective filing, or unexplained decision may demand individual examination even when the underlying decisions were produced in bulk.
Organizations respond to this imbalance by preserving the appearance of responsibility. The favored instrument is human-in-the-loop review, a close cousin to the human oversight requirement in the EU AI Act. A human decision-maker, reviewer, or supervisor remains formally assigned. As deployment expands, however, that individual may lose the practical capacity to meaningfully examine the system’s work.
Return to scientific publishing. The same cost collapse that lets a fabricated study pass for a real one increases the volume of material entering the publication system. Journals still depend on editors, reviewers, access to evidence, and eventual replication. The productive side of the system becomes scalable while its principal integrity mechanisms remain constrained.
Medicine reveals the same mismatch at the level of professional judgment. An imaging model may improve diagnostic capacity by comparing each scan with patterns learned from vast datasets. If the model also enables a large increase in case volume, the clinician’s nominal responsibility remains unchanged while the conditions needed to exercise that responsibility deteriorate.
Knowledge codification extends the problem from individual decisions to organizational assumptions. AI can draw patterns from manuals, databases, communications, and employee practices, making dispersed knowledge widely available. Greater reach also allows an error or hidden assumption to propagate throughout the organization before anyone recognizes it.
Agentic systems add a temporal dimension. Responsibility is ordinarily assigned at identifiable decision points. An agent may collapse several of those points into a continuous sequence of execution. By the time a person encounters the result, the relevant opportunity for intervention may have passed. Accountability must therefore be designed into the architecture of action: where review occurs, where authority enters, and where execution pauses pending judgment.
This architecture also determines whether AI genuinely empowers the people affected by it. Speech recognition may increase independence for someone with limited mobility. A legal tool may make unfamiliar procedures accessible. A customer-service chatbot may instead expand an institution’s processing capacity while reducing a customer’s access to someone authorized to resolve the problem. The relevant measure is the distribution of agency: who can act, contest, obtain an explanation, and compel correction.
The agency a person can exercise against a consequential decision measures the accountability the institution actually delivers. A decision that cannot be challenged leaves the affected person without agency and producing that decision cheaply and in bulk does not shrink what the institution owes for it. Cigna’s PXDX claim-review system, for example, reportedly let company physicians deny health-insurance claims in batches, spending an average of 1.2 seconds on each. Patients received a vaguely worded denial letter and entered an appeal process that required additional review before the decision could be disputed. Meaningful accountability therefore requires access to relevant evidence, authority to intervene, viable opportunities for contestation, and the institutional capacity to correct the process producing the outcome.
Whether those mechanisms keep pace can be measured through accountability elasticity, a term borrowed from economics: the degree to which an institution’s capacity for explanation, intervention, contestation, and correction grows as the system’s operational capacity grows. The lower the elasticity, the wider the lag. Each expansion then adds governance debt, activity the institution produces faster than it can meaningfully own.
No formula is needed to take the measurement. If use of an AI system increased one hundredfold, which mechanisms of accountability would increase with it? Who would review the additional decisions? Who could stop the system? Who could investigate a failure, correct its source, and answer to the person affected?
An institution without concrete answers has calibrated its oversight to a smaller and slower system than the one it now runs. That distance is the accountability lag, and it will not close on its own. The vendor ships a faster model and employees put it to use. Accountability grows only when someone builds it. No one is in charge is the inevitable result when an institution deploys the first without building the second.
***
Note: Accountability is one of 37 life cycle core principles. You can read more about it in the AI Life Cycle Core Principles Explorer.