Speech-to-speech systems must solve tasks whose requirements emerge across conversational turns. We introduce SpeechConversationBench (SCB), a focused evaluation of spoken mathematical reasoning using 103 sharded GSM8K problems. The framework compares the original problem delivered in one turn (full), its concatenated...
Injection is proposed, an inference-time intervention that combines explicit, role-specific capability guidelines with a guiding prefilled response prefix that improves role-capability alignment across models, reducing above-role accuracy by up to 0.562 while preserving in-role accuracy with a marginal drop of less tha...
Pakhapoom Sarapat, Saksorn Ruangtanusak, Kunat Pipatanakul et al.· 0 citations
SEATauBench is introduced, the first agent-focused evaluation framework for SEA sovereign AI that adapts Tau2-Bench to five languages and evaluates agents across progressively localized settings that vary the language of user-agent interaction, tool specifications, and task domains.
My Chiffon Nguyen, Aulia Adila, Saksorn Ruangtanusak et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.