Improving Evaluation of Heterogenous Congestion Control Algorithm Interactions
TL;DR
HarmGen is presented, a new tool that can efficiently explore a large search space using a genetic algorithm, to find network settings and workloads where there are poor interactions between heterogeneous CCAs.
Abstract
A critical part of new congestion control algorithm (CCA) proposals is an evaluation that a new CCA will reasonably share with already widely-deployed CCAs. However, the methodology for this evaluation is both inconsistent and inefficient due to the complexity of inter-CCA interactions under highly diverse network settings. We address these challenges by developing new metrics for evaluating fairness, developing an algorithm for determining when experiments converge, and applying this methodology to automated evaluation tool Mahak. In addition, we present HarmGen, a new tool that can efficiently explore a large search space using a genetic algorithm, to find network settings and workloads where there are poor interactions between heterogeneous CCAs. We show that with an identical budget of 300 experiments, HarmGen outperforms hill climbing and random search, and finds high harm values similar to a parameter sweep of 3500 experiments. With Mahak and HarmGen, we identify trends in BBR's evolution as well as issues with new L4S deployments.