Investigator: Zihao Li

Abstract:
The advent of Large Language Models (LLMs) such as the ChatGPT has ushered in a paradigm shift in societal functions, reshaping the way we think, create, and interact. Many believe that new LLMs are poised to reinvent the whole online ecosystem by redefining search engines and integrating various apps. While nascent LLM technologies have many advantages, new legal and ethical risks are also emerging. The accuracy of Generative AI is the one of the increasingly critical issues as LLMs become more widely adopted. Due to potential flaws in training data and hallucination in outputs, inaccuracy can significantly impact individuals’ interests by distorting perceptions and leading to decisions based on false information. Therefore, ensuring these models’ accuracy is not only a technical necessity but also a regulatory
imperative.

The EU is the first and foremost jurisdiction that has focused on the regulation of conventional AI models. However, the risks posed by the new generation of LLMs are likely to be underestimated by the emerging EU regulatory paradigm. While the EU Artificial Intelligence Act (AIA) established relevant obligations for general purpose AI, where LLMs are encompassed within its purview, the pertinent regimes are struggled to effectively regulate the hallucinations generated by LLMs. In contrast, with innovation driven by dominant US technology firms in generative AI and LLMs, the US government and technology industry are also taking action to ensure the trustworthiness of LLMs. As represented by Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, the US has responded to these technological advancements through industry-led initiatives and governmental directives.

This project will firstly examine the normatively acceptable limits of hallucination in LLM outputs, assessing the extent to which inaccuracy should be tolerated without compromising ethical and legal values. Secondly, it will critically compare and analyse the existing AI regulatory frameworks in the EU and the US, identifying gaps and inconsistencies in handling the hallucination challenges of LLMs. Lastly, this project will propose potential directions based on normative concerns, features of hallucinations and the entire lifecycle of LLMs, aimed at enhancing regulatory measures to ensure the ethical development and deployment of AI technologies.