B-187: Agreement Rates Between an Alternate Cutoff for the Test of Memory Malingering and Other Performance Validity Tests

Abstract

Objective: Determine concordance rates for Test of Memory Malingering (TOMM) Trial 2 cutoff of ≤ 49 (established by Erdodi & Rai, 2017) and other performance validity tests (PVTs) from multiple cognitive domains. Method/Data Selection: Participants were 137 patients (60.6% female; 63.5% white) that underwent neuropsychological assessment between 2010 and 2026 in an outpatient university doctoral clinical psychology training and research clinic (M/SD: age = 33.2/13.1; education = 14.1/2.5). Participants were administered the TOMM, as well as seven other PVTs from multiple cognitive domains including: attention/processing speed, language, visuospatial, memory, and executive functions. A PVT for motor functioning was also included. Concordance rates were computed for each PVT and the TOMM. Results/Data Synthesis: Agreement rates for TOMM Trial 2 cutoff ≤ 49 and other PVTs ranged from 79.2% to 87.8%. Reliable Digit Span had the highest concordance rate (87.8%). The lowest concordance rates were for Boston Diagnostic Aphasia Examination Complex Ideation (79.2%) and Finger Tapping Test (79.4%). More conservative kappa coefficients for TOMM and other PVTs ranged from -.02 to .38, in the poor to fair range, most within the slight agreement range. Conclusions: Even the more stringent single error cutoff for TOMM Trial 2 has high agreement rates with other PVTs from multiple cognitive domains in a clinical sample. Although kappa coefficients often are reported in PVT studies, they likely underestimate the actual agreement rate of PVTs because guessing is not a factor in PVT classification and the high PVT pass rate typical of most clinical samples inflates the estimate of chance agreement.