Auditing Artificial Intelligence Systems: A Survey of Current Frameworks, Principles and Approaches

Abstract

Over the past decade, the exponential integration of artificial intelligence (AI) systems across various sectors has been propelled by significant advances in machine learning algorithms, data availability, and computational power. This progress has produced highly effective AI systems, but also underscores the critical need for effective auditing to critically evaluate these technologies. In this article, we conduct a systematic review of the literature on methodologies, frameworks, and techniques for auditing AI systems, focusing on legal and ethical considerations and compliance with regulations. By reviewing key academic databases, including Google Scholar, IEEE, ACM, and Springer, we establish the scope of our survey and derive topics from our research questions. Our findings reveal gaps in current auditing practices and highlight the importance of incorporating AI value chain stages and AI maturity levels into auditing frameworks. This approach enables us to distinguish and recommend existing frameworks and methodologies that are most suitable for the specific contexts of different organisations, thus enhancing the effectiveness of AI system evaluations.