Post-update usability and user feedback of telepharmacy application: a mixed methods evaluation among general population and pharmacists

Abstract

Tanya Obat (translating to “Ask about Medications”) is an Indonesian telepharmacy application offering online consultations, drug information, and digital medication purchases. Building on a prior early-stage evaluation that found below-average usability, this study reassesses the updated platform at the national level to quantify usability gains and identify remaining user-specific barriers. Evaluating usability is critical for ensuring equitable access and sustained adoption of digital health tools. A concurrent embedded mixed-methods study design was employed, combining the System Usability Scale (SUS) with open-ended questionnaires. Participants included pharmacists and general population users across Indonesia. Associations between demographic characteristics and perceived usability between participants with below-average (SUS < 68) and acceptable (SUS ≥ 68) scores were analyzed using the Chi-square test ( p  < 0.05), with Fisher’s Exact Test applied when expected cell counts were below the acceptable threshold. Qualitative responses were analyzed thematically. A total of 423 participants were included, consisting of 189 pharmacists and 234 individuals from the general population. The mean SUS score was 66.3  ±  11.3 and 68.6  ±  10.9 for pharmacists and the general population, respectively. These results suggest that usability among pharmacists remained below the usability benchmark, while general population rated the application within the acceptable range. This difference was statistically significant (p-value = 0.031, 95% CI [-4.46, -0.21]). Experience in using smartphone applications was found to be significantly associated with perceived usability for Tanya Obat , with consistent associations across both pharmacists ( p  = 0.003) and the general population ( p  = 0.002). Qualitative feedback corroborated these findings, highlighting persistent issues with key concerns involving interface clarity, navigation, visual consistency, and feature functionality. The post-update evaluation revealed improved usability compared to the previous version, as reflected by higher mean SUS scores. However, technical enhancements alone are insufficient. Personalized design solutions are essential to ensure equitable user experiences and to sustain the application’s contribution to accessible, patient-centered pharmaceutical care in Indonesia. These findings provide practical guidance for telepharmacy design and implementation in low-resource settings.