RankNAS: A Pairwise Ranking Model to Accelerate Black-Box Neural Architecture Search

Abstract

Black-box Neural Architecture Search (NAS) is a robust, model-agnostic technique for creating application-specific neural network architectures. Regrettably, it is often prohibitively expensive as its many candidate architectures must be fully trained to be evaluated. To eliminate this overhead, we propose RankNAS, a lightweight pairwise ranking model that integrates into an evolutionary NAS framework at the candidate evaluation stage, enabling more efficient selection decisions independent of specific search or selection strategies. Contrary to other work, RankNAS encodes neural architectures using a handcrafted, fixed-length embedding rather than a graph encoding, resulting in a significantly simpler encoding that enables faster evaluation. We evaluate RankNAS on the CIFAR-100 dataset in both hardware-constrained and unconstrained settings under fixed low compute budgets of 3 and 10 GPU-hours. Predictor quality is first assessed in isolation: across 10,000 unseen pairwise architecture comparisons, RankNAS achieves ∼83% accuracy with an RMSE of 1.036 fitness units (relative RMSE 7.56%), with most errors occurring when comparing architectures achieving similar performance. The practical impact is then measured by integrating RankNAS into the NAS loop, thereby improving search efficiency and Pareto-front coverage. Overall, the results indicate that RankNAS provides both a reliable ranking signal and a low-overhead mechanism for accelerating black-box NAS under limited compute and hardware constraints.