Conversational AI for Public Administration Form Filling: Evaluating Agentic Architectures for Citizen–Government Interaction

Abstract

We present a systematic evaluation of conversational agent architectures for filling out administrative forms in European Portuguese. Three primary architectures are compared: a Vanilla LLM, a Fixed-Flow Multi-Agent Pipeline (FF-MAP), and a Planning-Driven Agent System (PDAS), including an enhanced variant with iterative self-evaluation (PDAS-F, where $F$ denotes the addition of self-evaluation feedback). Agents interact with simulated citizens to collect information and complete structured forms, with responses that may be partial, inconsistent, or linguistically varied. We assess performance across five forms of increasing complexity, using three GPT-5 family models (GPT-5, GPT-5-mini, GPT-5-nano) and automated metrics including total time, number of iterations, token usage, and hit rate (the proportion of form fields correctly filled). Our results show that PDAS generally outperforms the other evaluated architectures, achieving higher hit rates and more efficient interactions on complex forms within the tested dataset, with mixed-effects model estimated mean hit rates of 72.0% for PDAS, 66.8% for FF-MAP, and 53.4% for Vanilla LLM ( $p\lt 0.001$ ). The smallest model, GPT-5-nano, achieves competitive performance when embedded in PDAS, reaching a mean hit rate of 72.0% compared to 60.4% for GPT-5-mini and 57.4% for GPT-5 ( $p\lt 0.01$ ), suggesting that architectural design may play an important role alongside model size. Iterative self-evaluation in PDAS-F is associated with performance improvements; however, these gains appear to diminish after a few iterations. These findings suggest that, within the evaluated dataset, planning, task decomposition, and validation play an important role in building effective conversational agents, and indicate that small models can achieve high reliability when supported by structured architectures. The proposed framework and evaluation protocol provide a reproducible setup for conversational form filling in administrative contexts, and may offer preliminary insights applicable to other structured dialog domains.