privJedAI: A Unified Open-Source Pipeline for Privacy-Preserving Record Linkage

Abstract

Privacy-Preserving Record Linkage (PPRL) is a crucial task for interlinking different, but overlapping data sources without compromising their privacy. PPRL has evolved from simple hash-based exact matching methods to more sophisticated ones in the last two decades. However, existing open-source PPRL systems have limited utility, as they implement very few methods, typically of their creators, and thus, they do not facilitate users to build and benchmark different end-to-end pipelines. To address this shortcoming, this work introduces privJedAI, a new open-source library for PPRL that implements the Filtering-Verification framework, offering a plethora of state-of-the-art methods per workflow step. privJedAI leverages Python's data ecosystem to allow users to combine state-of-the-art techniques into end-to-end PPRL pipelines in an intuitive way that accommodates researchers and practitioners of any expertise. Special care is also taken to facilitate the benchmarking of the resulting pipelines to assess their trade-off between effectiveness and time efficiency across a series of established or custom datasets. We present the architecture of privJedAI, its unique features and explain how it can be used in practice.