Uncovering the Computational Ingredients of Human-Like Representations in LLMs
Evaluating over 75 models on a triplet similarity task finds that instruction fine-tuning and larger attention head dimensionality are among the strongest predictors of human alignment, while activation function choice, multimodal pretraining, and parameter size have limited influence on alignment.