Understanding Engineering Challenges in AI Agent Frameworks for Software Development: An Empirical Study of GitHub Issues

Abstract

AI agent systems increasingly support software engineering by extending large language models with capabilities such as planning, tool use, and coordinated execution, yet empirical evidence on the engineering challenges of building and maintaining such frameworks remains limited. To fill this gap, we conduct a large-scale empirical study of 3,864 closed GitHub issues from three representative repositories. We present a taxonomy of engineering challenges comprising 5 top-level categories and 21 subcategories, analyze the popularity and difficulty of these categories, and summarize 47 actionable solution strategies from resolved issue discussions and linked pull requests. These findings provide practical guidance for developers and framework providers, and offer an empirical basis for future research on AI agent engineering.