What Makes Usability Smells Detectable? On the Role of Representation in Interaction Log Analysis

Abstract

Automated usability evaluation using interaction logs offers a scalable way to identify potential usability problems. However, it remains unclear how different representations of interaction data influence the detection of usability issues. In this paper, we investigate usability smell detection from a representation-centric perspective. Using the AMUSED dataset, we decompose interaction logs into behavioral, structural, and contextual representations and evaluate their contribution to task-level smell detection with smell-specific gradient boosting models. We further compare learned models with rule-based detectors derived from prior work. Our results show that usability smells are representation-dependent and context-sensitive: no single representation performs best across all smells or applications and combining representations is beneficial only selectively. Learned models generally achieve more balanced precision and recall than rule-based detectors, while rule-based approaches remain useful for structurally explicit smells but depend strongly on available telemetry. These findings suggest that representation design should be treated as a central design decision in log-based automated usability smell evaluation.