When AI Governance Has to Prove Itself

Colorado’s proposed rules for automated decision-making technology (ADMT) and conversational AI treat AI governance as something an organization must prove. An organization should be able to reconstruct what its system did and identify the information and people that shaped an outcome. It should also provide a workable means of challenging that outcome and demonstrate that its safeguards still function after the system changes.

On August 11, 2026, the Colorado Department of Law filed the proposed Automated Decision-Making Technology & Conversational Artificial Intelligence Service Rules. The rules would implement Colorado’s revised ADMT Act and its new Chatbot Safety Act, both scheduled to take effect January 1, 2027. The proposal remains in formal rulemaking, and the Department is accepting comments through October 26, 2026, subject to extension if the hearing continues.

I set out here to examine the architecture underneath Colorado’s proposed requirements. Disclosure, human review, correction rights, age assurance, safety testing, incident reporting, documentation, and annual reports would all require the organization to retain enough knowledge about its deployed system to explain what happened and do something about it.

This analytical review was run through the AI Life Cycle Core Principles (AILCCP) framework. The legal obligations I cover come from Colorado’s statutes and proposed rules. The AILCCP adds a disciplined (i.e., repeatable) way of following those obligations across the AI’s life cycle and a systemic method of asking what an organization would need in place for them to work.

A note on capitalization: AILCCP principle names appear in initial uppercase, with the framework identifier in parentheses at first mention, as in Transparency (PR-033). Each principle is a defined term with its own scope, key questions, and mapped controls and standards. A lowercase term carries only its ordinary sense.

Consider, for example, data correction. Colorado’s proposal would require a decision to be reconsidered once the information behind it is corrected. Doing that may require the deployer to know what information was used, where it came from, what system configuration produced the original outcome, and whether that decision can be reproduced or independently revisited. Adverse-decision explanations likewise depend on traceability and human-review records; developer documentation affects what deployers can later explain; age-assurance requirements implicate privacy and retention choices; and product updates can alter safeguards that previously worked. Read together, the provisions operate across the AI life cycle rather than as discrete compliance requirements.

The AILCCP also kept forcing questions of proof. For each requirement, I asked what an organization would actually need to produce if challenged. Could it identify the configuration that produced a particular decision? Recover the data used? Identify its source? Show who reviewed the outcome and what authority that person possessed? Produce the testing that supported a safety claim? Demonstrate what changed after an incident?

Broad principles such as Transparency (PR-033), Accountability (PR-002), Human-Centered (PR-017), Privacy (PR-023), and Safety (PR-029) translate into specific evidentiary demands when framed this way. For Transparency, that may mean reconstructing an individual decision. For the Human-Centered principle, an independent review process with access to primary evidence. For Safety, evidence that a control continued to perform after a model or product update.

The analysis kept arriving at one question: What end-to-end governance system does an organization need to demonstrate, test, explain, contest, correct, and improve the behavior of its deployed AI? The answer begins with the ability to reconstruct what happened.

The AILCCP also operates as a dynamic restatement. It identifies the norms emerging from legislation, regulation, standards, and enforcement as defined principles, and the framework is refined as those sources develop. Colorado’s proposal proves to be such a source. I used the AILCCP to test whether the proposal made certain governance capabilities concrete enough that the framework should name them expressly, and that exercise produced concepts such as Decision Reproducibility, Independent Review Firewall, Continuous-Interaction Risk Assessment, and Feasibility as an Evidence Claim. They are now candidate refinements for the framework, and the analysis behind each runs through what follows.

Decision Reproducibility

Under Colorado’s proposal, a person who receives an adverse outcome from covered ADMT would be told what produced it. The deployer would have to disclose the system’s role, the involvement of human reviewers and other systems, the principal reasons for the outcome, relevant inferences or scores, certain factors that automatically produced denial, and other information relevant to the decision. A deployer would fail the explanation requirement if it could not explain how the ADMT materially influenced the decision or could not accurately explain the principal reasons for the adverse outcome.

A system model card may describe how a model generally operates, but a person challenging a particular decision needs the record of that decision, and for many contemporary AI systems that record extends well beyond “model version.” The meaningful unit of reconstruction is the configuration that produced the outcome, meaning the system’s complete operating state at the moment of the decision. The AILCCP surfaces that unit through Accountability, which asks whether an output can be traced to the data, models, versions, and owners behind it. In a contemporary system, the configuration can span the model and checkpoint that were running at the time, the system prompt, retrieval sources, feature flags, classifier versions, third-party data, scoring thresholds, orchestration logic, and the rule governing when the matter was sent to a person.

This changes explainability from merely a communications exercise into a deployment capability and potential competitive differentiator. The AILCCP anticipates that shift in Explainability (PR-012), which pairs explanations with the data quality and provenance controls behind them. If a system cannot preserve enough of its operating state to explain an individual outcome, the failure predates the adverse decision.

The same preservation problem runs through Colorado’s data-correction provisions, which contemplate access to the personal data used in a consequential decision, including individual scores, classifications, predictions, recommendations, inferences, and inputs. When inaccurate data is corrected, reconsideration means running the decision process again with the corrected information. Where possible, an adverse outcome would also be stayed pending correction or meaningful human review.

A conventional database correction may be inadequate in that setting, because reconsideration can require the historical context surrounding the original decision. A continuously updated system, for example, may produce a different result when run weeks or months later because the model, retrieval corpus, threshold, or surrounding workflow has changed. Re-running today’s system does not necessarily tell us what corrected information would have done to yesterday’s decision. The organization must therefore preserve enough of the decision environment to revisit the outcome properly or maintain an independent means of reconsidering it when faithful replay is no longer feasible.

Independent Review Firewall

Reconstruction, however, is only useful if someone with authority can act on what it shows. That someone is the human reviewer, and Colorado is mindful of how easily “human in the loop” becomes an inert label rather than a safeguard. The proposal would attach substance to the person’s role by requiring, where feasible, a reviewer who is independent of and not subordinate to the original decision-maker, possesses subject-matter understanding proportionate to the consequences, receives appropriate training, and has actual authority to approve, modify, or override the decision. The proposal would also shield the reviewer from management pressure and retaliation, protections that keep the required independence from eroding in practice.

A person may technically participate in a process while having little practical ability to alter it. A reviewer who sees only the machine’s recommendation, lacks the underlying evidence, is measured primarily by throughput, or understands that deviations are unwelcome may add a human signature without supplying meaningful review. The AILCCP probes that failure through the Human-Centered principle, which asks whether oversight is sustainable and what prevents automation bias. A credible review process therefore also turns on what the reviewer sees, what the affected individual can submit, and how the reasoning behind the final result is preserved.

A system that recommends an outcome anchors the reviewer, pulling independent judgment toward whatever the machine already concluded, and that risk explains why the proposal would bar ADMT from assisting in the meaningful human review. A tool used only to retrieve records, translate material, provide accessibility support, or organize a file presents a different issue. The proposal’s anti-anchoring objective is understandable, but the boundary between decisional assistance (shaping or recommending the outcome) and bounded clerical assistance (retrieving, translating, or organizing material) will need to be carefully defined in the final rules.

The evidence a reviewer or a deployer needs is often created elsewhere, because a modern AI system combines the work of several companies. A foundation-model developer may supply the underlying model. Another company may adapt or integrate it. A deployer may add proprietary data, prompts, and business rules. Other vendors may provide identity information, scores, retrieval sources, or safety tools. The affected person, however, sees one system.

Colorado’s proposal responds by requiring governance information to move downstream. Its provisions concerning developers and midstream developers call for documentation of intended and inappropriate uses, categories of data, known limitations and risks, monitoring, relevant factors, and use restrictions. Midstream developers, in turn, carry obligations for the developer documentation they obtain and transmit.

At each handoff, someone should know what documentation was received, what remains unknown, which uses are permitted, what conditions affect reliability, which version is being supplied, when changes will be communicated, and who is responsible for preserving downstream evidence about actual outcomes. The AILCCP frames these requirements through Data Stewardship (PR-005), which asks whether provenance is tracked end-to-end. Vendor assurances about “responsible AI” carry little weight against that question. A deployer facing a contested decision needs usable evidence, and if it disappeared somewhere between developer, integrator, and deployer, governance failed at the handoff.

Continuous-Interaction Risk Assessment

The chatbot provisions shift the focus from single decisions to sustained interactions. A safety test built around individual prompts and outputs misses what a conversational system can do across time. Memory, personalization, and repeated interaction may gradually produce a relationship that a single exchange cannot capture.

In evaluating whether a service falls within provisions concerning emotional companionship or emotionally dependent interaction, the proposal looks to features including anthropomorphic personalization, assigned identity attributes, memory of earlier exchanges, sycophantic or continually validating behavior, engagement-reinforcement mechanisms, and whether safeguards remain effective after model or feature changes.

Any one of those features may be benign in one product design and consequential in another, and the way those features compound across days or weeks may matter more than any single feature on its own. Testing should therefore follow representative interactions over time and after material changes to the system. The AILCCP presses one step further through Consent (PR-006), which treats consent as valid only while the user’s understanding still matches how the system actually behaves. A relationship that deepens over weeks can outgrow the understanding the user gave at the first exchange, and consent obtained then may not cover what the service has since become.

Feasibility as an Evidence Claim

Nowhere is the obligation to keep testing more visible than in the treatment of technical feasibility, which Colorado would evaluate with reference to available technologies, comparable industry safeguards, alternatives, periodic reassessment, realistic testing, performance after updates, and documentation of rejected approaches. A design choice justified by the state of technology in January may deserve another look in July.

A metric can be calculated correctly and still create the wrong incentive, and the proposal’s crisis-referral reporting for conversational AI is the clearest example. Operators would have to report the number of referrals relative to conversations and the proportion that actually involved suicidal ideation or self-harm risk under an evidence-based method. The obligation would extend to detection and response protocols, safety measures, testing, reliability and efficacy over time, age estimation, and interactions involving minors and prohibited sexual content.

Suppose a system becomes more conservative about issuing referrals. The proportion of referrals associated with genuine risk might improve because borderline cases are no longer referred. At the same time, the system could be missing more people who actually need intervention.

That is why governance needs paired measures. The AILCCP devotes a principle to the problem, Metrics (PR-020), which warns that optimizing what is measured can degrade what is not. Referral accuracy should be read alongside measures that expose what it cannot show, among them missed-risk rates, detection sensitivity on a governed evaluation set, false positives, time to escalation, performance changes after updates, and, where appropriate, variation across languages or interaction contexts. Measurement choices are themselves controls, and they warrant the same scrutiny as any other safeguard.

Taken together, Colorado’s proposal points toward an understanding of AI governance that is considerably more operational than a collection of policies, principles, or one-time assessments. On that understanding, a deployed AI system should leave enough evidence to answer questions about a real event. Which configuration produced the outcome? Could the decision be revisited once inaccurate data was corrected? Did the reviewer possess genuine authority? Did a safety control continue to work after the model changed? Did the user’s consent still cover what the service had become? Could the deployer obtain what it needed from the developer? These are difficult questions to answer with the policies and one-time assessments written before launch. Each asks what the system actually did after launch, and only records kept while it was running can answer that.

Any safeguard, whether mandated by Colorado or defined in the AILCCP framework, is effective only if the organization can use it when something bad happens. Each AILCCP principle turns that condition into something testable, defining a capability, pairing it with probing questions, and identifying the evidence that proves the organization has it. The framework took Colorado’s long list of requirements and rendered it into capabilities an organization either has or lacks, which recasts compliance as an infrastructure question rather than a provision-by-provision chase. Someone will need an explanation, so decisions must be traceable. A decision will need to be revisited, so outcomes must be reproducible. An affected person will need a genuine second judgment, so the reviewer must be independent. And once the system changes, yesterday’s safety evidence no longer describes it, so the safeguards must be retested, and their design reevaluated, after every material change. Those are the capabilities Colorado could codify. And because the AILCCP distills legislation, standards, and enforcement from many jurisdictions, an organization that builds them for Colorado is building them for whatever comes next.