ierfair: An R Package for Ability-Conditioned Fairness Diagnostics of Insufficient-Effort Responding on Background Questionnaires.

Abstract

Insufficient-effort/careless responding (IER) on self-report background questionnaires attached to large-scale assessments (e.g., PIRLS, PISA, and TIMSS) is routinely screened with indicator-based methods (e.g., longstring, response variability, Mahalanobis distance, and psychometric synonym/antonym consistency). These indices are computed from a respondent's own response pattern and implicitly treat any inconsistency as inattention. We show this assumption is not fairness-neutral: lower-ability respondents may produce more inconsistent-looking patterns simply because background-questionnaire items are harder for them to process, independent of genuine effort. We introduce ierfair, an R package that benchmarks standard IER indices against ability-stratified (or local-regression) reference bands built from an external achievement score, flagging a respondent only when their index is extreme relative to others at the same ability level. Applying ierfair to the PIRLS 2011 Indonesia Student Background Questionnaire (N = 4,720), we confirm a significant ability confound in three raw indices (r = -.15 to .25) that per-index conditioning removes but find that a natural rule for combining several conditioned indices ("flag if ≥ k of m fire") can reintroduce a significant residual ability trend even when no individual index shows one. Requiring unanimity or switching to local-regression conditioning reduces this trend, and pooling indices into a single score before thresholding, implemented as iec_composite (), removes it most cleanly, keeping the flag rate within 1-2 percentage points of the sample mean across every stratum. The package requires no response-time data, an important constraint for archival large-scale-assessment data, and produces an audit-trail report summarizing flag rates across ability strata and demographic subgroups.