Purpose

Many of the Core Principles (second column) are compiled from work done by the G7, OECD, UNESCO, IEEE, ISO, NIST, FTC, G20, and APEC. Other Core Principles, such as Big Data, Consent, Fidelity, Metrics, Permit, Track Record, and Wherewithal are my additions.  The third column (What it means and aims to promote) is comprised of mostly of my analysis. My objective in this third column is to diminish the inherent ambiguity in these Core Principles.

While ambiguity may initially seem like (to borrow from software parlance) “a feature, not a bug” in that it accommodates more latitude for interpretation, it is not; it is a bug. Ambiguity around the Core Principles fuels a persistent and stubborn lack of precision, a definitional vacuum. It destabilizes stakeholder ability to develop and maintain a cohesive and rational discussion around the core principles. This, in turn, hampers outcome predictability in the sense that laws, regulations, standards, and best practices that refer to the Core Principles become more vulnerable to ambiguity which then renders them less, or entirely, ineffective. Wiping away the distortive effects of ambiguity allows for more efficient, universal use of the Core Principles. They can serve as practical attributes that all stakeholders can hone-in on and leverage for virtually all aspects in which they engage with AI. For example: Developers can select Core Principles that apply to their application and measure where their work is aligned and where it isn’t; end users can reference the Core Principles in their application licensing efforts, in their due diligence, and maintenance of the application; regulators can use the Core Principles to better guide their enforcement activities; and law makers can use them in drafting laws that are more relevant, clear and practical.

Finally, the term “life cycle” here is intended to emphasize the continual assessment and management character of these principles. The OECD’s Framework for the Classification of AI Systems defines the “life cycle” as “planning and design; collecting and processing data; building and using the model; verifying and validating; deployment; and operating and monitoring.” (The NIST AI Risk Management Framework also closely follows this definition.) My approach here is a bit broader and includes the all activities related to model decommissioning. A life cycle approach to the application of the Core Principles is essential for their efficient application. Applications, after all, are not static. They typically undergo numerous updates and upgrades, not all of which are always beneficial. Similarly, end user environments are not static. The scope of acceptable use can change as can its leadership team. (For more on this, take a look at the Governance principle.)

Standards identified in green are published and those identified in orange are in development.

Notes

This section is devoted to an on-going discussion that aims to promote substantive understanding of the Core Principles.

1. GPTZero (https://gptzero.me) and similar tools enhance understanding of AI outcomes, operation, accountability, and transparency. As such, it belongs to the XAI Core Principle.
2. Incorporating durable life cycle features to protect against bad actors is likely going to be a feature that the FTC will scrutinize. This will be particularly important for OpenAI given their findings in the GPT-4 System Card.
3. The Permit principle (also) considers and influences how LLM training occurs. From a legal perspective, some of the key training issues to consider are copyright and contract law. The Permit principle promotes adherence with the Data Stewardship Framework in that it helps ensure that training occurs only with data that complies with the Permit principle.
4. The Permit principle is as a key feature of the AI regulatory framework. It helps mitigate harm by rendering more efficient the assignment of liability and compensation mechanisms.
5. The Cooperation principle includes information sharing. This activity is formalized through, for example, participation in Information Sharing and Analysis Organizations (ISAOs) which were set up under the Cybersecurity Information Sharing Act (2015). Participation in these types of organizations is a cybersecurity best practice and is mentioned in the NIST Cybersecurity Framework. ISAOs are an effective way for members to collaborate on identification and dissemination of information about cybersecurity threats. And a similar set up can be adopted for mitigating risk in AI. This organization could take a name such as “AI Information Sharing and Analysis Organization” (AI-ISAO) and provide a forum for members to share risk information and risk mitigation practices.
6. The Permit principle relates to policies and procedures that center on formal certification frameworks. These frameworks allow the development and use of high risk AI applications only to those that have successfully passed formal requirements and continuously maintain such certification. The “formal” attribute reflects the identity of the permit issuer: a government agency. For example, regulatory agencies such as the Office of the Comptroller of the Currency (OCC), Food and Drug Administration (FDA) and Federal Communications Commission (FCC) issue various operational permits based on applicant disclosures and the agency’s independent assessment. So long as the applicant meets the agency’s operational criteria, the permit is granted and remains valid subject to continued compliance with that agency’s rules. The same approach can be used for AI, but the regulatory framework would also be applicable to end users. For this group, a permit would be issued only to end users that demonstrate the required operational proficiency and are legally obligated to maintain it. This also brings us to the question of whether it is necessary or desirable to have a new AI-centered regulatory agency? Or do we already have sufficient regulatory capabilities for AI within the OCC, FDA, FCC, etc?
7. Among the requirements for Transparency is that algorithm documentation and annotation (e.g., a README file) follow established best practices. This is important for compliance with other Core Principles, such Audit and Reliability. Documentation and annotation best practices may also differ based on the coding language that was used. In Python, for example, the documentation and annotation methodology is set in the Python Enhancement Proposal (PEP), which is periodically updated. It remains to be seen how PEP (and other documentation) requirements will evolve to accommodate the regulatory and lawmaking needs.
8. The Trustworthy principle can be further understood through the May 4, 2023 announcement by National Science Foundation (NSF) regarding the funding of seven new National Artificial Intelligence Research Institutes. The Trustworthy AI in Law and Society Institute (TRAILS) is led by the University of Maryland in partnership with George Washington University, Morgan State University, and Cornell University. TRAILS examines four key “thrusts” in the AI development lifecycle. The following is my brief interpretation of each of these thrusts: (A) Social Values: System output syncs with community and other stakeholder values; (B) Technical Design: Transparency, trust, robustness and (what I see as) XAI are key development principles in algorithm design;  (C) Socio-Technical Perceptions: Again, an emphasis on developing XAI; and (D) Governance: Development of documentation and analysis of the performance of governance regimes. Another vector for understanding Trustworthy is the NIST U.S. LEADERSHIP IN AI: A Plan for Federal Engagement in Developing Technical Standards and Related Tools: “Trustworthiness standards include guidance and requirements for accuracy, explainability, resiliency, safety, reliability, objectivity, and security.”
9. The human-in-the-loop aspect of the Human-Centered AI principle needs to be considered in context of the application; not all AI applications need to equally have it and not all applications need it. This brings us to the question of which applications need it the most? Applications that are categorized as having a high potential of societal/ethical impact should have (in varying degrees) humans making final decisions. For example, AI applications used in employment, credit scoring, law enforcement, surveillance, military defense, immigration, asylum, and border control should have the highest. Applications that do not share this impact profile need less of or no human involvement. For example, personal assistants such as Siri or Alexa, image and voice recognition, fraud detection, and predictive analytics. The human-in-the-loop requirement can also be evaluated through the AI Utility Levels Schema. For example, Level 4 applications are more likely to have the potential for high societal/ethical impact than Level 1 applications and, therefore, should have greater human involvement. Within the context of the EU’s taxonomy, Critical AI is more likely to require more attention to enabling human-in-the-loop than applications belonging to the Non-critical category. This analysis is important for a variety of reasons, for example, guiding an AI audit and assigning liability.
10. One of building blocks of the Ethics principle calls for AI development to be focused on applications that are deemed “socially beneficial.” This means that the AI application development goes through gate reviews that test for: (i) discrimination; (ii) exclusion; (iii) misinformation; (iv) malicious use (e.g., hate speech, deep fakes, other psychological harm); and (v) UX/UI harm (e.g, UI that is intentionally or reasonably likely to be deceptive). As we consider these elements, it is also important to keep in mind their dynamic nature. This is to say that each of these is susceptible to augmentation based on the power/capability of the underlying AI. For example, the likelihood of misinformation from an LLM trained on 100 million parameters is likely lower than one trained on 100 billion parameters. Stated differently, the augmentation power of the latter to produce output that looks credible (in terms of sentence structure, tone, key words, and content)–but isn’t–is likely higher than that of the former.
11. International Organization for Standardization (ISO) AI standards set is provided under ISO/IEC JTC 1/SC 42. Currently, the Trustworthy ISO family contains seven sections (one administrative and six substantive). What makes for Trustworthy AI under ISO can be understood from how it maps to the Core Principles. Some of this is straightforward as we can see a 1:1 relationship to the Core Principles, meaning that they use the same name. They are: Bias (24027) and Explainability (6254). The others are: Ethics and Social Concerns (24368); Controllability (8200); Beneficial AI Systems (21221), and Treatment of Bias (12791). (The parenthetical is the section number.)
12. User consent to the application’s objectives requires providing a legally-valid user consent interface. Due to the potential operational complexities involved with AI, part of this task needs to feature an appropriate notice. Without carefully executing this, the likelihood of obtaining legally-valid user consent significantly fades. Signs of this clarity requirement can already be seen, for example, in the White House Blueprint for AI Bill of Rights, specifically the Notice and Explanation function. It calls for the provision of “clear, timely, and accessible” documentation that’s provided in “plain language,” with “clear descriptions” of how the AI system functions, what it’s intended for, who is responsible for it, and that is periodically updated so that it remains current. One method for achieving this is through the AI Fact Label which I described in the December 10, 2019 update to the Artificial Intelligence and Computational Law: Democratizing Cybersecurity post. The AI Fact Label can be delivered to the end user in a number of ways, for example via a pop-up notice (which aligns with the “timely” and “accessible” requirements).
13. Safety considerations should be analyzed throughout the development cycle in a formal manner. This can be accomplished, for example, through adopting the AI Risk Ratio (ARR). Originally, the ARR was proposed as a risk scoring method for IoT devices based on looking at the relationship between the IoT device’s AI power (application type), its proliferation (user base/threat surface, use frequency and/or duration of use) and the risk it poses. But any type of risk can be evaluated this way (the original version looked at privacy) and where the application’s score breaches the organization’s risk tolerance, its development is ended.
14. Ethics ranks as one of the top issues in AI development and use. Because it is intrinsically a broad topic, it is prone to various interpretations that can result in diverse approaches to execution. Contributing to this diverse treatment are, for example, geographical and cultural variances. The variables can have a destabilizing effect on the universal applicability of this Core Principle, in terms of the weight it is given.
15. Part of the Wherewithal core principal refers to the developer’s policies and procedures that fully support the design of Core Principle compliant AI. An essential component here includes an examination of the developer’s application warranty. All too often, there is a dissonance between the marketing side of the business and the details provided in the terms and conditions. The former offers all kinds of benefits, including, for example, reduced cost and enhanced productivity. The terms and conditions often present a very different picture. Here we see broad stroke disclaimers used in an attempt to help shield the developer from any liability. Knowing this, a developer can be assigned a higher degree of compliance with the Wherewithal core principal where the application warranty language it provides is more in sync with the promises it makes on the marketing side. An end user engaged in a due diligence analysis of the application should, therefore, make sure to also examine the warranty language and grade the developer’s compliance with the Wherewithal according to its findings. Of course, other factors in the Wherewithal core principal should be evaluated and graded as well so that a higher quality assessment can be completed.
16. Within the Accountability core principle we see the role that a periodic audit function can take. One of the functions undertakes identifying LLM vulnerabilities throughout the model’s lifecycle. This is important because, for example, possible model integrity degradation (e.g., data poisoning) can occur as more models are added. As audit frequency tends to positively impact the reduction of error or failures of the AI system, the organization’s alignment with the Accountability core principle grows.
17. Compliance with the Privacy principle is a multi-dimensional undertaking. In addition to federal and foreign jurisdiction frameworks (e.g., HIPAA, GLBA, COPPA, and GDPR) we currently have in the U.S. eight state-level privacy laws and 12 more working their way through the legislative process. All this activity makes the task of maintaining compliance with privacy law more complicated because the legal requirements around the protection of  personal information are not uniform. That said, there are things developers and end users (each from their own vantage point) can do. For developers, focus can be placed throughout the development gating process on key privacy themes centered around data subject rights, namely the right to: access, correct, delete, receive notice, and consent (opt-out/opt-in). And for end users, focus on effective contracting practices with developers can ensure that these data subject rights areas were formally part of the model/application development and deployment cycle.
18. One of the components in the Privacy principle calls for conducting a privacy impact assessment (PIA). The privacy impact assessment (PIA) is a process for assessing and ensuring compliance with applicable legal, regulatory, and internal policy privacy requirements. The law does not prescribe how to conduct this assessment, leaving companies free to use whichever method is reasonably designed to yield the proper results. Companies that have a track record of doing business in the European Economic Area, Switzerland, or the United Kingdom are already familiar with the PIA, as this is a requirement under the General Data Protection Regulation (GDPR) and similar laws. In the United States, however, this is a relatively newer requirement. Currently, some U.S. state privacy laws (California, Colorado, Virginia, and Connecticut) have begun embracing the PIA, but others (such as Utah) have not. Making the PIA part of the organization’s ongoing privacy-bolstering processes makes sense even when it is not legally required. Some of the benefits in doing so include increasing operational efficiency, lowering the company’s risk profile, and generally enhancing public trust in your company’s handling of PII likely outweigh the costs.
19. An application that is compliant with the Consent principle continuously maintains alignment with the end user’s consent. This continuous feature helps ensure the developer/licensor’s liability profile remains stable and that the application’s functionality remains steady with the end user’s expectations. Though consent is sometimes regarded as a single, check-the-box act given by an end user before commencement of certain activities, this is not always the case. In applications that operate in highly regulated domains such as healthcare, end user consent should be sought on multiple occasions depending on, for example, the changing nature of the activity. Continuously maintaining alignment with the end user’s consent, therefore, means that the application is designed to prompt the end user for new consent anytime there is an expected functional deviation from the parameters that governed the previous transaction.
20. The Explainability (XAI) principle is malleable. To be meaningful, the approach used to implement XAI depends on the identity of the audience. When directed to end users, the method will need to be simple, doing the best to avoid undesirable situations such as a TLDR scenario. It should be different when directed to enterprise end users or similar sophisticated audiences where a more detailed approach is desirable and may even be contractually required.
21. A useful example of the Transparency principle in action is available in the Washington State Bill 5152. In relevant part, it provides: “An electioneering communication which contains synthetic media may not be distributed without a disclosure. The disclosure must state that the media has been manipulated and: (i) for visual media, be printed in at least the largest font size of other text in the media or a size easily readable for the average viewer; (ii) for video media, appear for the duration of the video; and (iii) for audio media, be read in a clearly spoken manner and a pitch easily heard by the average listener at the beginning and end of the audio, and at least every two minutes during the audio, if applicable.” Note that SB 5152 is focused solely on the notice elements of the Transparency core principle. There are additional important elements that should be considered to render this law more effective. For example, requiring that developers and deployers of these systems demonstrate that they align with: the Do No Harm variable found in the Safety core principle, and the compatibility with democratic values variable in the Human Centered core principle. Requiring a license (part of the Permit core principle) should also be considered as it helps mitigate the risk of bad actors using synthetic media in sensitive applications such as elections campaigning.
22. The Transparency principle has 11 variables, which is more than any of the other Core Principles. Among the variables found in this Core Principle is the use of techniques that deliver effective notice and explanation. Their primary aim is simple: preventing end user harm caused by unfair or deceptive actions. Considering the significant amount of regulatory chatter that broke out after the unveiling of OpenAI’s ChatGPT, one question that frequently arises is whether the regulatory gap is as big as it is made out to be or maybe not so much? The answer is that it depends on what the focus is on. When it comes to regulating unfair or deceptive practices, there are already laws that deal with that. Most notably on this stage we find Section 5 of the Federal Trade Commission Act, a section that has been extensively and broadly used by the FTC to curb these practices. Looking back at the FTC’s pretty successful track record in tackling cross sector incidents under Section 5, it is likely that it will also do well when it comes to AI. That said, it is likely that over time AI-specific refinements will be made by the FTC so that companies are more in tune with what it is focused on when regulating unfair or deceptive actions.
23. Among the variables that make up the Ethics principle are development policies and procedures that create, enhance, and maintain stakeholder trust in the application and those that demonstrate commitment to develop socially-beneficial AI applications. When it comes to the use of synthetic media, holding true to these ethics variables can become challenging. This is partially due to the relative ease in which this type of media can be created and also the relatively high degree of potential damage (e.g., deepfakes). Ultimately, media is comprise of data and this is where the Data Inventory Controls in the AI Data Stewardship Framework come into play as these controls help ensure that the creation and use of synthetic media is consistent with promoting the trust and social-beneficial application variables.
24. The model’s risk profile is a variable that is managed under the Security principle. This task includes evaluating the code and model for the degree of vulnerability to a prompt injection attack, misinformation, and misalignment with the Human Centered principle. The evaluation should be conducted periodically. Its frequency will mostly depend on which party has the leverage to require it. Parties should also consider formalizing the requirement in a service level agreement (SLA) or similar contract. The SLA is also a good place for detailing the type of analysis and format of findings.
25. Information sharing is one of the key characteristics of the Cooperation, Robust, Security, Sustainable, and Transparency core principles. This is particularly important in protecting against the Business Identity Compromise threat vector, which has been the subject of repeated warnings from the FBI. In a BIC, fraudsters imitating legitimate employees can cause “very significant financial and reputational impacts to victim businesses and organizations.” And as deepfakes become more complex, they get more difficult to identify. Information sharing is one way to counter this threat and it can be augmented through effective practices that focus on build a deepfake ontology of all known…how should I put it…”fakers.” Using this approach, AI deepfake detectors will be able to reference this ontology to identify the likelihood any given media has been altered before it is viewed. Whether the deepfake is flagged or deleted becomes a user-based setting that can also become part of the AI Risk Ratio (discussed here).
26. Machine unlearning (MU) is part of the Ethics core principle. It is a promising technique aimed at addressing disparate impact, discrimination, and preserving privacy. The primary objective of “unlearning” is to identify and eliminate specific data from the training model while erasing its influence. However, this endeavor is not without its challenges. For the removal process to hold legal significance, it must consider potential vulnerabilities like membership inference attacks. Despite the data removal, this method of compromise involves querying the model to predict whether the removed data was originally present in the training model. A successful attack determines that a specific individual’s data was present, undermining the MU process.
27. Transparency practices that align with the Ethics core principle involve providing information that is accurate, sufficient, and useful. Note that the “accurate” element within the Transparency core principle is heavily dependent on the degree of compliance with the AI Data Stewardship Framework. The degree to which there are no (or insufficient) controls around data accuracy, the relevant entity’s credible ability to claim that it complies with Transparency is compromised. As to the “useful” element, this is not merely a question of whether the information is relevant. It is also important to make sure the information can be acted upon in a meaningful way and without negative consequences. For example, providing a disclosure by a company that AI is being used along with an opt-out option that does not penalize the end user. Deficiency in any of these three elements works to deteriorate alignment with both core principles and this can damage an organization’s contractual and other obligations, such as those created by its terms of use and privacy policy.
28. The Permit core principle can apply domestically or internationally. Achieving the latter is relatively more complex to implement as it requires a higher degree of agreement and ongoing coordination among member countries.
29. The Security core principle maps to the Privacy principle in a number of ways. For example, the degree to which a model is vulnerable to a model inversion attack, property inference attack, or a membership inference attack creates a risk of misalignment with the Privacy principle. The degree of vulnerability to these attacks is also a question that will be assessed in the Data Incident Response Plan (see the AI Data Stewardship Framework). One of the incident response tasks calls for identifying the attack vector. Here, the analysis will include, for example, a review of whether the incident was the result of a white-box attack (where the adversary has full access to the model) or whether it was a black-box attack (where the adversary has limited access to the model).
30. An AI system’s Reliability profile is composed of multiple elements. Continuous validation and the use of proven risk assessment methodologies are some of the key features that build this profile. Red teaming is an example of a well-known and accepted risk assessment methodology. It has existed well before AI made its grand entry to the world’s stage in the Fall of 2022 and searches for system vulnerabilities and weaknesses in the same way a malicious adversary would. The use of red teaming helps make the AI system more operationally reliable and can also serve an important legal function. It can be useful, for example, in demonstrating to a regulator that the developer used reasonable procedures to identify flaws in its AI system. In a contractual setting, the existence of red teaming can help support the type of warranties that a licensor provides, potentially allowing for more robust offerings that help distinguish it from those of its competitors that don’t use this methodology.
31. Accountability is a model for identifying and assigning liability. It’s an interdependent AI life cycle core principle in that it enables other core principles such as Bias, Equity, Ethics, Human Centered, and Reliability. However, Accountability can be vulnerable to inefficiency where the amount of resources required to achieve it are not used in an optimal manner. We can see this come into play when we look at how responsive an AI system is to legal demands. Suppose an AI system causes harm. The current method of dealing with it is manual, through human intermediaries. It usually involves identifying the harm, communicating it to a lawyer who then communicates it to the AI operator who then, if responsive, takes steps to remedy the harm. This process can be automated; perhaps not for all types of harm, but in some. For example, a demand that an LLM developer remove and unlearn certain content (a takedown demand) is amenable to automation. This does not mean that the results will always be 100% accurate, but it can be a way of efficiently providing appropriate remedies to individuals/entities that claim their content was unfairly or illegally used. This process could be dealt with internally by an LLM developer, offering it as a feature of their service. But more compelling is the idea of a centralized process, a clearinghouse-like model that LLM developers voluntarily participate in and in return enjoy a safe harbor from content-use claims of liability. One of the key features of this clearinghouse is that it would be an AI system, which leaves open the question of management. Ultimately, it would be fully autonomous, but in the early versions it would be managed through an organization setup by an international treaty. In terms of protecting the system from sham demands, one way to accomplish this could be to require a takedown applicant to make a deposit that signals that their demand is made in good faith.
32. The Permit core principle drives many of the key aspects for meeting the Safety core principle. This relationship makes high-risk applications and high-powered AI systems (in terms of computational power) the prime candidates for a development and use permit. There are two things I want to highlight here: First, this double-permit approach, which requires that the developer and end user both have a permit for their respective uses helps maximize AI application and AI system compatibility with other life cycle core principles such as Reliability and Security, which in turn help maintain alignment with the Ethics core principle. (This also serves to show just how permeable the core principles are with each other.) Second, consider the relationship of the Permit and Safety principles to the AI Risk Ratio (ARR). This ratio models the relationship between computational power and output: the greater the computing power of an AI application is, the greater the probability the application is capable of producing highly beneficial and highly harmful output. (For additional discussion on this capability, take a look here.) The ARR is another way to think about the permit requirement, especially as it relates to high risk applications and high-powered AI systems.
33. Timely detection of and response to cybersecurity threats and incidents of compromise (especially, but not limited to data) is a key characteristic that makes up the Security core principle. What constitutes as “timely” is driven by a variety of factors and it is important to choose the right one(s) in order to stay within the Security swim lane. One way to determine how to set a timely response is to identify relevant regulations and work backwards to identify what’s required to enable that. Take for example a publicly traded company that suffers a cybersecurity breach. The company is subject to the regulations of the Securities and Exchange Commission (SEC) and it needs to take into account the relevant reporting requirements that are set by the SEC. As it stands, the SEC’s cybersecurity rules require disclosure of breaches that are considered “material” within four business days from when that determination was made. Four days can be a daunting timeframe, but knowing this requirement enables the consideration, planning and execution of putting in place personnel, policies and procedures that enable compliance. This, in turn, enables alignment with the Security core principle.
34. Unpredictability is not necessarily bad; it can be a bug and it can also be a feature. This duality creates some tension with maintaining system alignment with the Predictability core principle as the default thinking around it is to see deviation (any deviation) as bad. In certain settings, unpredictable system behavior is beneficial, however. In the context of scientific research, for instance, an AI system’s capability of quickly identifying unconventional patterns or relationships in complex datasets can lead to highly valuable advancements that would otherwise would be impossible. The same is true for AI systems used in game development, creative art and design. Additionally, it is important to be mindful of the degree of deviation from the Predictability principle. A minor deviation that results in minor harm does not deserve the same amount of attention as one that causes significant harm. It’s also relevant to consider whether the overall benefit of the system’s potential for unpredictable behavior outweighs the potential harm. This assessment, however, gets more complicated as it raises potential tensions with maintaining system alignment with the Ethics core principle.
35. AI certification serves an important component of the Permit core principle. A development and operation license (read: the permit) can, for example, be conditioned on maintaining certification by a well-known, proven organization such as ISO. In this context, the certification becomes mandatory, whereas in other settings it is obtained by the developer/end user as a way to gain a market advantage. Whether it is obtained under a voluntary or mandatory setting, certification can be seen as an additional security layer that promotes the catchall Trustworthy core principle. But not every AI application requires a permit and not every AI application should be the subject of certification. Level A applications, for example, are unlikely candidates for a permit, but the developer may still want to get certified as a way to gain a competitive edge.
36. Explainable AI (XAI) is an AI life cycle core principle that enhances other life cycle core principles, such as Accountability, Reliability, Fairness, Ethics, and Transparency. XAI can be viewed as a key compliance feature for AI applications that require output explainability. For example, for organizations that extend credit based on algorithmic decisions, the Consumer Financial Protection Bureau requires that they be able to explain instances in which credit was denied.
37. Ongoing analysis of all aspects of design, development, and deployment is a key requirement in the Govern function of NIST’s AI Risk Management Framework (AI-RMF). We see this requirement in wording such as “mapping, measuring, and managing AI risks” (Govern 1 and 2) and in “collect, consider, prioritize, and integrate feedback” (Govern 5). In order to align with the requirements of the AI-RMF, organizations need to ensure they also align with life cycle core principles such as Accountability, Accuracy, Fidelity, Metrics, Relevant, Reliability, and Transparency.
38. Making available an AI system card serves to align with the Ethics life cycle core principle. The system card also plays an important part in promoting the Transparency core principle (a subset of Ethics). Given the important role a system card plays in informing the end user about the inner workings of the AI being used, it can be seen, more broadly, as a must-have feature, one that end users should demand of developers prior to signing up for the service. Now let’s shift gears and consider the role of the system card in the context of foundation model-based applications used by lawyers. And this brings us to an interesting question: Should lawyers be required to be familiar with the system card their application uses? There’s a good argument in the affirmative. Let’s take a look at how this works. The ABA Model Rule of Professional Conduct 1.1: Competence provides that: “A lawyer shall provide competent representation to a client. Competent representation requires the legal knowledge, skill, thoroughness, and preparation reasonably necessary for the representation.” Furthermore, Comment [8] to the Rule provides: “To maintain the requisite knowledge and skill, a lawyer should keep abreast of changes in the law and its practice, including the benefits and risks associated with relevant technology, engage in continuing study and education and comply with all continuing legal education requirements to which the lawyer is subject.” [Emphasis added.] The system card contains information that help the lawyer align with the requirements under the Rule and Comment [8]. The system card provides information that is not available elsewhere in the application. Not referencing the system card prior to use of the application could be seen as falling short of the Rule requirements.
39. Transparency is one of the most complex core principles in that it contains numerous characteristics. Among these is disclosing the model’s dataset’s provenance and its legal status (with regard to its licensed or unlicensed content). This presents a conundrum for GenAI developers because their foundation models use massive amounts of data that, for the most part, no license was ever sought or granted. This presents a bigger challenge than initially meets the eye. Since most of the core principles are interdependent (there is hardly a principle that stands alone) non-compliance with Transparency has a repercussive effect; one failure breeds many more. The challenge, therefore, is not merely whether the GenAI developer infringes on copyright (they most likely do in the U.S.) but that they also derail with respect to their compliance with, for instance, the Accountability and Fairness core principles. And there’s more. GenAI developers are not the only ones facing difficulty because of this. It also negatively affects end users. The reason this unfolds this way has to do with how GenAI developers mitigate their risk. Many of them dilute their contractual obligations to the end user to the point where the terms and conditions are nearly (if not entirely) diametrically opposed to the marketing materials. The latter make grand promises about the application’s capabilities, the terms and conditions all but wipe them out. Under these conditions, a GenAI developer’s compliance with the Transparency core principle is largely left to market forces. But in an environment where GenAI is pretty much dominated by a few mega corporations (Meta, Google, OpenAI/Microsoft), this doesn’t leave much hope for end user’s getting a better deal anytime soon.
40. Effective XAI enhances compliance with the principles of Accountability, Reliability, Fairness, Ethics, and Transparency. XAI facilitates stakeholder trust. But XAI is not a principle that independently drives trust. It can deliver for the user the “why” and “how” of the output, but not necessarily that that the output is accurate. This is where the AI Data Stewardship Framework (AI-DSF) kicks in. Because it helps ensure the input is accurate it increases the likelihood that the output is as well and this helps augment XAI.
41. The AI, Algorithmic, and Automation Incidents and Controversies (AIAAIC) collects data about “incidents and controversies” caused by AI. According to the AIAAIC, incidents in the “privacy” category represent the highest risk in AI. This suggests that developers and end users carefully consider how their operations align with the Privacy life cycle core principle. Within this core principle is the requirement that organizations use methods that are compatible with and maintain the state of anonymized, pseudonymized, and encrypted data. Checking the box on this requires putting together a robust cybersecurity regime. There are many sources to reference for attaining that. A few notable examples: NIST Cybersecurity Framework (CSF), ISO/IEC 27001 and the CIS Critical Security Controls (formerly, the “SANS 20”) which is a subset of the NIST SP 800-53, which also informs the NIST CSF. Another source to consider is the California Consumer Privacy Protection Agency (CPPA) as it provides instructive (though at this point non-final) cybersecurity guidance relevant to organizations that use “automated decision-making” technology, which essentially means AI. Though the CPPA is tasked with implementing the California Privacy Rights Act (CPRA), which amends/extends the California Consumer Privacy Act of 2018 (CCPA), what it has to say about cybersecurity holds true even for organizations that are not subject to the CPRA. Furthermore, taking into account multiple cybersecurity references in the design of one’s own cybersecurity regime can be thought of as a mapping exercise which typically includes standards, guidelines, and frameworks within its scope. At the end of the day, mapping is an exercise of finding concepts that relate to each other. As most of the cybersecurity sources mentioned earlier contain similar concepts, mapping them can enable a better understanding of how they interrelate. This ultimately promotes attaining a robust cybersecurity regime and, in turn, better alignment with the Privacy principle.
42. Trustworthy is a catch-all life cycle core principle; one that is frequently used but often remains unexplained. The table above provides a view of the elements that comprise the Trustworthy core principle, some of which are themselves standalone principles. ISO 22989-2022 (Information technology – Artificial intelligence – Artificial intelligence concepts and terminology) takes a view similar to what is provided in the table above. It maps Trustworthy to the core principles of Bias, Fairness, XAI, Robust, Reliability, Resilience, Predictable and adds “controllability,” which is not a standalone core principle above but is part of the Human Centered core principle. ISO 22989-2022 is also a bit different than the AI standards set in ISO/IEC JTC 1/SC 42 in that it does not call out the Ethics core principles as being part of Trustworthy.

43. Maintaining a robust cybersecurity regime is relevant to measuring the extent to which an AI developer’s work aligns with the Security life cycle core principle. One method of generating a useful measurement of this is through the use of a thorough cybersecurity due diligence checklist. The organization looking to integrate the AI application (the end user) should spearhead this task. In doing so, it should ensure that any gaps (risks) are satisfactorily dealt with prior to signing the agreement. One method to follow in this process involves the use of a comprehensive cybersecurity checklist. Experienced cybersecurity attorneys will likely be familiar with and use a robust questionnaire that examines the relevant aspects of the developer’s operations. These aspects can be grouped as follows: (i) Policy, Certification and Best Practices; (ii) Asset Management; (iii) Access Control; (iv) Network Security; (v) Endpoint Protection; (vi) Vulnerability and Patch Management; and (vii) Incident Response. Each of these sections contains detailed questions and tasks that need to be reviewed with the developer. For example, the Policy, Certification, and Best Practices section directs inquiry into whether the developer’s organization has an information security officer, whether it has a current information security policy, and whether employees and subcontractors are required to read and sign it on an annual basis. Once completed, the results of the due diligence should be reviewed by appropriate management representatives and by the organization’s legal counsel. The latter will use the results to negotiate and draft appropriate terms and conditions into the service agreement that will help the organization reduce its risk.

44. Governance contains a duality character in that the principles it is comprised of apply equally to the developer and the end user. Since the proper maintenance of the Core Principles requires effective on-going governance efforts by both parties, any failure in doing so has the potential to undermine alignment with other Core Principles. For example, an AI developer’s failure to sync its organizational policies, processes, and procedures with the product’s design and functionality tends to weakens its product’s alignment with Accountability, Accuracy, Equity, Fairness, Predictable, Privacy, and other Core Principles.
45. Executive Order (EO) 14110 on the Safe, Secure, Trustworthy Development and Use of Artificial Intelligence aims to formalize responsible use and governance of AI. A review of the EO’s contents and structure sheds light on the weight attributed to the Core Principles. Safe appears 58 times and its corollaries “secure” and “security” appear 18 and 126 times, respectively. (“Secure,” and “security” are not separate Core Principles.) Privacy scores 37 times and its corollary “private” 12. Bias scores 12 mentions. On the lower end of the spectrum, Trustworthy appears 7 times, which is really just 6 times considering that it also appears in the EO title. In terms of requirements, according to a Stanford University Human-Centered Artificial Intelligence review of the EO, Safe carries 30 requirements and Privacy 9. All said, Trustworthy ekes out the least amount of emphasis in 14110, which leaves the impression that the EO is mostly concerned with the Safe and Privacy Core Principles. This is partially reminiscent of the EU’s AI Act. There we see the most significant emphasis on “risk.” In it, the term “risk” appears 456 times and “high risk” 301. Taken together, the EO and the EU AI Act appear to place the most weight on the Safe Core Principle. It is also possible to glean from this EO that the Administration places significant emphasis on the Safe, Equity, Human-Centered, Fundamental Rights, and Privacy Core Principles. While the EO applies to entities that do business with the federal government, it can be expected that a similar emphasis will be replicated in state government and municipal procurement and use.
46. The risk of proliferation of fake content creates additional pressure of compliance with a number of Core Principles. Take, for example, the Accountability Core Principle. It is one of the most important principles to align to in the battle against fake content. It instructs, among other things, that the application’s output needs to be traceable to the appropriate responsible party and that the application is responsive to legal demands. These capabilities are also important for creating and maintaining end user trust. For a deeper dive on how the Accountability Core Principle ties in with trust, consider the January 4, 2024 update to the Captology post.
47. New York’s “Acceptable Use of Artificial Intelligence Technologies” (NYS-P24-001) highlights the role of an “Information Owner.” (Sections 4.1 and 4.2.) The policy does not define what this title means, so it is unclear, for example, what type of qualifications this person must have. The policy does provide that this person is tasked with “periodically assessing the outputs of their in-production AI systems to validate continuing reliability, safety, and fairness.” These two sections reflect  Accountability and Governance requirements.
48. The Ethics principle can be thought of as the basis for building an organization’s “ethics brand” as it relates to the development and/or usage of AI. Pretty much any organization that develops and/or uses AI can benefit from committing to creating and maintaining an AI ethics brand. The brand identifies and broadcasts the organization’s alignment with the Ethics principle. It does this for internal-facing stakeholders, such as employees, shareholders, and board members. It also has an external-facing character. In this regard, the AI ethics brand is (primarily) aimed at customers and competitors; but it does not necessarily need to stop there. It and can also be leveraged to project onto other potentially relevant diverse audiences, including, for example, regulators and lawmakers. The AI ethics brand is embodied in the organization’s policies and procedures. From there it propagates into all relevant operations. And in addition to signaling the organization’s alignment with Ethics, the AI ethics brand demonstrates the organization has a culture that embraces continuous learning and improvement. This ideally results in differentiating it from competitors and strengthens its defensive stance, staving off potential litigants.
49. The AI ethics brand (see note 48) should reference, as necessary, relevant elements of well-established procurement frameworks. For example: The U.S. Government Accountability Office (GAO) “Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities” GAO-21-519SP and IEEE P3119 “Standard for the Procurement of AI and Automated Decision Systems.” Referencing these frameworks in the organization’s alignment efforts with the Ethics principle imbues the resulting policies and procedures with authority. This helps the organization present a solid, defensible posture to its stakeholders and in public settings, such as litigation.
50. Algorithmic bias presents a variety of challenges. One of these is its degrading effect on the Transparency principle. Algorithmic bias is a bug; it is vulnerable to being poorly understood by developers and end users alike. This vacuum in understanding distorts alignment with the Transparency principle. This situation is like developing a drug where the side effects are not known, perhaps only partially known, and the side effects are not entirely or appropriately disclosed. It communicates an attitude of: ‘Go ahead and use this, but we and you don’t know exactly what we’re getting ourselves into and neither of us may like what happens next.
51. As AI-powered legal applications get better and have a firmer, deeper integration in the practice of law, lawyers will increasingly find themselves in the role of AI operators and supervisors. Our focus as lawyers will be ensuring that there is optimal input and carefully examining the output before it is delivered to the client or court. (Note: The “input” ties to the AI-DSF.) So, rather than replacing lawyers, AI can be seen as changing/redefining some of the tasks we do. And all this gets us closer to the concept of requiring licensed AI operators and supervisors. Two primary drivers for this are the rules of professional conduct and the terms and conditions that govern the use of these applications. In California, for example (and this holds the same for other states as well), lawyers are required to continuously learn about AI, its abilities and limitations and review all output. Doing so is particularly important in light of the fact that the developers/licensors of these applications usually accept no responsibility for their application; it’s something that becomes quickly apparent from the terms and conditions.The concept of a licensed AI operator and supervisor is aligned with the Permit life cycle core principle. Think of it as operationalizing the requirements under the rules of professional conduct. As licensed AI operators and supervisors, the lawyers are better aligned with meaningful compliance.
52. In a move reminiscent of the Future of Life Institute’s AI “pause letter,” TechCrunch reports on yet another AI “open letter.” This one, like the letter issued in March 2023 (which called for a six-month pause in the development of AI models more powerful than GPT-4), was signed by hundreds of well-known AI community leaders. Their focus is on deepfakes and they call for its “strict regulation” and controlling the “deepfake supply chain.” The letter identifies eight “reasons” why action should be taken: “Nonconsensual pornography; fraud; elections; practicality; urgency; inadequate laws; mass confusion; and performers.” Some of the reasoning in the letter makes sense, but others like “practicality, inadequate laws, and performers” are, to say the least, a poor fit and it seems like they were shoe-horned in. Another ill-fitting feature in this letter is an acknowledgement that “not all signers have the same reasons for supporting” the letter “and they may not all agree on its content.” In any event, like the March 2023 letter, this initiative is also doomed; it will not yield any new laws nor will it energize enforcement beyond that which already exists and would have been applied anyway. Of course, none of this is to suggest that clamping down on the proliferation of deepfakes is not important as it certainly is. The point is that these types of letters are fundamentally a waste of time. The signatories’ effort on this topic would be much more effective if it aimed at highlighting and promoting less obvious but nonetheless important and practical initiatives such as, for example, strengthening information sharing policies and procedures. Note also that information sharing helps organizations align with the Cooperation, Robust, Security, Sustainable, and Transparency core principles, all of which help in the fight against deepfakes.
53. AI developer vetting is an important process that should be completed by the end user prior to deploying the application. This task is especially vital for applications that have a high-risk latency characteristic, such as those targeted for use in health care, legal services, etc. Part of this effort should examine the AI developer’s alignment with the Governance principle. For example, questions should be directed at the vendor’s use of a formal data management framework such as the AI Data Stewardship Framework. Presuming that the vendor follows such a framework, the next step is to set formal commitments that are based on it. This part is done in the agreement that is entered into between the parties. Here, the end user can introduce a provision where the vendor warrants that its data licensing practices are routinely and regularly reviewed, updated, and enforced and that there will be no deviation from this. There should also be a provision that if there is such a deviation, it will amount to a material breach and grounds for immediate termination plus appropriate damages, including, but not limited, liquidated damages.
54. XAI needs to be aligned with the Accountability, Reliability, Fairness, Ethics, Trustworthy, and Transparency principles. This is important keep in mind because XAI on its own can be misleading. Why? Because an explanation is only good if it meets the criteria espoused in these principles. If it doesn’t, the explanation becomes a method to drive the end user towards a specific outcome that is beneficial to the developer and not aligned with the end user. Stated differently, the explanation is rendered hollow if it doesn’t meet those principles.
55. Adobe’s Content Credentials enables creators using the company’s AI features to add metadata to their content. It’s a mechanism similar in concept to the AI Fact Label and aligns with the Transparency principle. Content Credentials is part of other Adobe Transparency-related initiatives: The Content Authenticity Initiative (CAI) and the Coalition for Content Provenance and Authenticity (C2PA). This is all good, but CAI and C2PA are useful standards only where the creators are good actors. They fall apart when the actors are not. Bad actors will either not use these standards or hack them into manipulative ends. So long as AI-generated content is hosted on a relatively small platform, like Adobe’s Creative Cloud, it is easier to enforce compliance with CAI and C2PA. But how other online platforms, such as Facebook, Instagram, TikTok, etc., can (or are willing to) enforce the use of these and/or similar standards, remains to be seen. At the end of the day, for such metadata labeling to be effective, a certification-like mechanism needs to be used across (at least) all major online platforms; something similar to SSL/TSL website certificate. This effort will require preventing the upload of content that doesn’t have the CAI/C2PA or similar standard certification. An online platform provider that refuses to participate in this framework should be deemed non-compliant with the Transparency principle. Legislators and courts should then take a dim view of such non-compliance and ensure appropriate legal remedies against such actors are swiftly dispensed.
56. Anthropic claims that its latest GenAI app, Opus, “exhibits near-human level of comprehension and fluency on complex tasks…” Really? So, let’s take a closer look at what Anthropic really says about its app. We start by looking at their terms and conditions of service. In relevant part we find the following:YOUR USE OF THE SERVICES AND MATERIALS IS SOLELY AT YOUR OWN RISK. THE SERVICES AND OUTPUTS ARE PROVIDED ON AN “AS IS” AND “AS AVAILABLE” BASIS AND, TO THE FULLEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW, ARE PROVIDED WITHOUT WARRANTIES OF ANY KIND, WHETHER EXPRESS, IMPLIED, OR STATUTORY.Forget all the claims you read earlier. Those were just meant to get your attention and subscribe to the service. What really matters and what Anthropic is really saying is that your use of Opus is at your own risk, it doesn’t stand behind Opus and there’s no legal theory that you will be able to avail yourself of if something happens.This type of disclaimer has been around for a very long time and is commonly used. But when it comes to AI apps (and robotics), a different approach is necessary. AI is vastly different than any other technology development in (at least) the last 100 years. There is wide consensus that AI represents a technological leap that is, on the one hand, dramatic as the introduction of electricity and, on the other hand, challenging to deal with as nuclear weapons. Securing broad social agreement on what constitutes responsible development and use of AI and making sure all stakeholders work with this framework is, therefore, essential. AI developers should not be allowed to use such disclaimers.
57. The European Parliament overwhelmingly approved today (March 13, 2024) the EU AI Act – 523 in favor, 46 against, and 49 abstentions. With its emphasis on a “human-approach” to regulation of AI, the AI Act takes the view that proper regulation of AI means that it must be aligned with fundamental human rights and places significant emphasis on humans-in-the-loop. As such, the AI Act is poised to challenge and test many operational aspects of companies that come within its scope. And here it is likely that we will see significant attention placed on the Governance core principle. Thus, companies subject to the AI Act will need to ensure they maintain alignment with this principle. Pretty straightforward? Maybe not; doing so may ultimately prove more difficult than initially anticipated. Why? Because as a matter of default, AI development is aligned with the developer, not the end user. Absent appropriate controls this can complicate efforts to comply with this new law. Therefore, companies will need to quickly figure out how to remain laser focused on the Governance core principle not only in their AI development efforts, but also for the entire AI product’s life cycle.
58. Utah’s Artificial Intelligence Policy Act (AIPA) was signed into law on March 13, 2024 and goes into effect May 1, 2024. Among the provisions is a requirement to disclose that  generative AI is being used if asked. The disclosure must be verbal “at the start of an oral exchange or conversation; and through electronic messaging before a written exchange.” The timing of the disclosure is suspect. It’s unclear what would prompt someone to inquire about the use of generative AI before the start of the oral exchange or conversation and before a written exchange occurs. In any event, enabling this type of disclosure requires that the organization maintain alignment with the Ethics, Transparency and Governance core principles.
59. A newly proposed law, the AI Consumer Opt-In, Notification Standards and Ethical Norms for Training (AI CONSENT) Act, will require a consumer’s express opt-in consent before the consumer’s data may be used for training an AI system. The AI CONSENT Act calls on the FTC to be charged with specifying the disclosure standards and how consent needs to obtained in order to be valid. (If this law passes, it is likely there will be disputes on whether valid consent was obtained.) Similar to the approach used in privacy laws across the country, to be valid the disclosure will need to be easy to understand, easily granted and/or revoked, and a consumer’s election cannot be used to affect the services provided. Aside from the obvious alignment with the Ethics core principle, the AI CONSENT Act also aligns with Accountability, Consent, Fairness, Fundamental Rights, Governance, Human-Centered, Privacy, Reliability, Sustainable, Transparency, and Trustworthy core principles.
60. Terms and conditions that disclaim all liability for the system’s operation are incompatible with most of the Core Principles. Consider the following common disclaimer, copied from a developer of a popular AI chatbot: “CUSTOMER’S USE OF THE SERVICES ARE AT ITS OWN RISK. DEVELOPER DOES NOT MAKE, AND HEREBY DISCLAIMS, ANY AND ALL OTHER EXPRESS, STATUTORY AND IMPLIED REPRESENTATIONS AND WARRANTIES, INCLUDING, BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT AND TITLE, QUALITY, SUITABILITY, OPERABILITY, CONDITION, SYSTEM INTEGRATION, NON-INTERFERENCE, WORKMANSHIP, TRUTH, ACCURACY (OF DATA OR ANY OTHER INFORMATION OR CONTENT), ABSENCE OF DEFECTS, WHETHER LATENT OR PATENT, AND ANY WARRANTIES ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.” This language fails the Trustworthy core principle (among others) in that it erodes public trust and confidence in the AI system.
61. The NIST AI Risk Management Framework (RMF) places a healthy degree of emphasis on the Human in the Loop core principle. So does California’s Office of Information Security Generative Artificial Intelligence Risk Assessment SIMM 5305-F (5305-F). Among the quality and safety controls found in the 5305-F Part III Generative Artificial Intelligence Risk Assessment is the requirement that “The GenAI system will have human verification to ensure accuracy and factuality of the output.” The RMF provides additional clarity on this. It recommends that “a human operator or user is notified when a potential or actual adverse outcome caused by an AI system is detected.” It also suggests that governance (another core principle) is an essential driver for enhancing the performance and quality of human oversight. There is a glaring question that is currently not dealt with, however: To what extent does the human oversight requirement offset the benefits of using a generative AI system? Since the RMF and 5305-F demonstrate strong alignment the Human in the Loop principle, the amount of resources that will need to be directed to human oversight are likely to be significant. And in this setting lurks a risk that I refer to as “operational fatigue.” In the beginning, when the AI system is new, there is a great deal of energy and devotion to building a Human in the Loop mindset. But as time goes on the amount of human oversight required to maintain it can create a drag effect. This is where the processes and procedures that support the human oversight actions begin to fray. The solution to this is careful monitoring, which also aligns with the Metrics core principle. Ultimately, it is better to stop using a generative AI system that is operationally burdensome than using it in a manner that is misaligned with the Human in the Loop and likely other principles, such as Accountability.
62. GenAI applications do not “understand” the input/prompt nor the output. The application has no idea if it’s lying, hallucinating, or providing relevant/satisfactory output. It doesn’t “know” anything. Its output is based on complex statistical probability calculations of language syntax. Even where reinforcement learning with human feedback (RLHF) is used to fine tune the application, the application still does not “understand.” This is important for attorneys to understand and internalize. Why? Because this is an essential ethical requirement, part of fulfilling the Competence requirement under the ABA Model Rule of Professional Conduct (RPC) Rule 1.1 and Comment [8]. Knowing that GenAI applications do not “understand” also demonstrates alignment with the Ethics core principle. And yet, even now, even after cases such as Mata v. Avianca, some attorneys remain under the impression that GenAI applications can “understand.” This is not a trivial oversight. It can easily lead to a lax attitude to the due diligence requirement under the RPC and derail alignment with the Ethics core principle.