No. 112: AI Systems and a Risk of Discrimination in Creditworthiness Assessments: The Role of the EU Artificial Intelligence Act in Safeguarding the Right to Non-Discrimination

Abstract

This thesis deals with one of the most prominent risks stemming from the use of AI systems – the risk of discrimination. It focuses on the risk of discrimination posed by AI systems used for the purpose of assessing creditworthiness. This area of AI systems’ deployment is sensitive from the perspective of fundamental rights, as it impacts the lives of individuals to a great extent. The aim of this thesis is to assess the existing legal framework that safeguards the right to non-discrimination in credit scoring assessments conducted with the use of AI systems. The thesis first analyzes the legal framework consisting of the European non-discrimination laws, the General Data Protection Regulation (GDPR), together with two judgements of the Court of Justice of the European Union (CJEU) and the 2023 Consumer Credit Directive. Secondly, this dissertation assesses how the EU Artificial Intelligence Act (AI Act) complements this legal framework. The AI Act qualifies AI systems used for assessing creditworthiness as high-risk. Consequently, the providers and deployers of these systems are required to comply with certain requirements. This thesis is narrowed down to the analysis of the provisions of the AI Act that are most relevant for mitigating the risk of discrimination, including a provision dealing with the Fundamental Rights Impact Assessment (Article 27 of the AI Act), human oversight (Article 14 of the AI Act), bias mitigation and monitoring (Article 10(5) of the AI Act) and individual-level rights (Articles 86 and 85 of the AI Act). The prominent goal of the thesis is to understand the role of the AI Act and its added value to the existing legal framework safeguarding individuals’ right to non-discrimination in creditworthiness assessment. In addition, this thesis discusses whether the systems used by traditional credit agencies, such as SCHUFA, fall within the scope of the AI Act. The thesis also touches upon the challenges that AI systems pose to the European non-discrimination laws, and it contributes to the debate on the existence of the right to explanation in the GDPR.

Details

Author(s):
  • Julia Małgorzata Kalus
Publish Date:
May 8, 2025
Publication Title:
European Union [EU] Law Working Papers
Publisher:
Stanford Law School
Format:
Working Paper
Citation(s):
  • Julia Małgorzata Kalus, AI Systems and a Risk of Discrimination in Creditworthiness Assessments: The Role of the EU Artificial Intelligence Act in Safeguarding the Right to Non-Discrimination, EU Law Working Papers No. 112, Stanford-Vienna Transatlantic Technology Law Forum (2025).
Related Organization(s):

Other Publications By