Beyond Price and Benchmark: A Cost–Methodology–Fit Framework for Selecting AI Developer Tools, with a Proposed Evaluation Protocol

Abstract

The market for AI software-development tools has expanded faster than the frameworks used to evaluate it, leaving practitioners to choose among code-completion assistants, AI-native integrated development environments (IDEs), and terminal-native agents on the basis of headline price or benchmark rank — neither of which predicts realised value. This paper makes two contributions. First, it develops the Cost–Methodology–Fit (CMF) framework, an analytical model that treats tool selection as the alignment of a team's dominant workflow with a tool's interaction paradigm and billing structure, grounded in the established SPACE model of developer productivity [1]. Second, drawing on a systematic documentary comparison of leading tools (verified June 2026) and on the conflicting experimental literature — a controlled trial reporting a 55.8% task speed-up [2] against a randomised trial of experienced developers reporting a net slowdown [3] — it derives the central claim that AI-tool value is workflow-contingent, not tool-intrinsic. Because documentary comparison cannot establish causal productivity effects, the paper additionally specifies a reproducible mixed-methods evaluation protocol that adopting organisations can run to measure fit in their own context. We report the framework and protocol, not new empirical outcomes, and state this scope explicitly. Findings indicate the market has bifurcated by billing model, that capability is increasingly a model-level rather than tool-level property, and that hybrid tool stacks are a rational response to fit-contingency. Keywords: developer productivity; SPACE framework; usage-based pricing; agentic development; evaluation protocol; tool selection