Addressing gaps in breast cancer risk prediction scores in the electronic health record.

Abstract

758 Background: Breast cancer risk prediction scores, such as the Gail Model, are commonly implemented in the Electronic Health Record (EHR) to guide clinical decision-making regarding breast cancer screening and chemoprevention. Yet, their implementation is inconsistent or limited to specific time periods or populations. Further, scores can drastically change for an individual patient over time as personal and family history evolves or due to errors in implementation, potentially leading to biases in research analyses if used uncorrected. To address these gaps, we investigated the feasibility of computing the Gail score based on EHR data, including extracting the required variables for risk score calculation, and compared these scores with those available in the EHR as a part of routine care. Methods: Initial investigations indicated that the Gail score was only implemented for some women post-2018 at Geisinger, a large integrated healthcare system in northcentral Pennsylvania. We extracted all variables needed to compute Gail scores from the EHR, such as age at menarche, age at first birth, race, family history, and biopsy information. Using several note-based sources in the EHR, we developed basic natural language processing (NLP) pipelines to extract breast cancer family history information for first degree relatives and biopsy information. 50 dual chart reviews were conducted to assess performance of variable extraction. Using all extracted variables, we computed Gail scores for each patient-date combination with an existing EHR Gail score, using data available up to that date, and compared them with scores recorded in the EHR. Results: Chart reviews demonstrated high accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the NLP pipelines (Table 1). Gail scores were computed for 93,088 unique patient-date combinations that had scores available in the EHR. The mean squared error (MSE) between these scores was 0.41% for 5-year predictions and 7.49% for lifetime predictions. The mean absolute error (MAE) between computed Gail scores and those available in the EHR was 0.29% for 5-year predictions and 1.41% for lifetime predictions. Conclusions: Cancer risk prediction scores in the EHR should be investigated for potential biases before research use. Computing Gail scores based on EHR data is feasible and can mitigate limitations due to the implementation of these risk scores. Selected performance characteristics of features created via NLP pipelines. Statistic Family History of Breast Cancer Atypical Hyperplasia in Breast Biopsy Accuracy 95.6% 100.0% Sensitivity 91.3% 100.0% Specificity 96.3% 100.0% PPV 81.3% 100.0% NPV 98.4% 100.0%