Jul 2026· International Conference on Conversational User Interfaces· pp. 1-10· 0 citations· 41 references
Computer Science
TL;DR
A user study comparing two modalities for writing prompts for generative AI tasks reveals that input modality significantly influenced prompting behaviour but did not lead to measurable differences in subjective evaluations.
Abstract
Generative AI systems, such as ChatGPT, are increasingly integrated into how users work and create content. The most common way to interact with these systems is through text-based prompting, typically produced by typing on a keyboard or using speech-to-text. While speech input is supported in generative AI systems, little research has examined how speech and typing differ for prompt creation. We conducted a user study (N = 40) comparing two modalities for writing prompts—speech and typing—across two common generative AI tasks: image creation (e.g., a self-portrait) and text creation (e.g., a blog post). We examine behavioural metrics (prompt length, time, and iterations) and subjective perceptions of the experience and generated output. Our results reveal that input modality significantly influenced prompting behaviour but did not lead to measurable differences in subjective evaluations.
Text-to-music (TTM) generation systems allow users to create music through natural language prompts, yet it is unclear whether the descriptive language used to prompt aligns with descriptive language used to summarize or describe heard music. We pair 200 real-world Udio prompts with their generated audio and free-form descriptions collected from English- (n = 70) and Korean-speaking (n = 78) listeners, and contribute a human-derived taxonomy of musical prompting vocabulary grounded in real user data. Using this framework, alongside word- and vector-level analyses, we find a consistent structural asymmetry: prompts are dominated by Genre and Story/Narrative language. Genre terms propagate most reliably from prompt to perception, while narrative-heavy prompts are the strongest predictor of semantic misalignment. A preliminary cross-cultural comparison further suggests that description profiles vary across listener populations along narrative, functional, and affective dimensions, raising questions about whether current TTM systems, trained on aggregated English-centric corpora, can accommodate the full diversity of how people naturally express musical ideas.
Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional transcription and restructuring from AI-backed dictation tools. How do they impact an LLM's performance? In this paper we present HIVE (Human Input-Variation Engine), a suite of voice transcription perturbations and QWERTY keyboard perturbations. We use HIVE to evaluate how robust models are to these perturbations. We present seven findings. (i) Voice transcription perturbations lower accuracy across every instruction-tuned model we test, and it is the structure of the transcription rather than its fillers that carries the cost. (ii) QWERTY keyboard perturbations cost less, and a model absorbs a lot of them before accuracy falls away. (iii) Both trace back to one cause, how many of the question's tokens survive the perturbation: destroying a token is what hurts, while adding new ones alongside it costs little. (iv) The gap between the two channels appears only where the answer must be constructed or deduced; on multiple choice there is none. (v) The harm does not solely come from test-set contamination. (vi) It cannot be trained away with lightweight adaptation. (vii) A thinking budget recovers the keyboard channel almost entirely but leaves the spoken registers untouched, and compressed speech is worse with it.
Zizhao Hu, N. Segura, Mohammad Rostami et al.· 0 citations
Large language models (LLMs) are increasingly integrated into computing education, yet nearly all prior research has focused on text-based interactions. As voice-enabled interfaces become more capable and more common, there is growing interest in understanding how voice input might shape students' use of LLM-powered tools. In this exploratory study, we investigated how introductory programming students interact with Prompt Problems, which are programming tasks that require crafting natural-language prompts to generate correct code. Students (N = 919) solved a series of Prompt Problems with the freedom to select or switch between text and voice input modalities. We collected their prompt submissions as well as post-activity survey responses, then analysed differences in prompt accuracy, persistence, and perspectives by modality. For two of the three problems, we found that students who typed their prompts using text were more likely to have those prompts succeed on the first attempt than students who submitted unedited voice prompts. There was no difference in success rate if students edited their transcribed voice prompts before submission. Across the problems, we found evidence that students who tried voice prompting varied in their usage of modality – perhaps indicating a complementary, or non-preferential approach. However, most students only tried and reported preferring text. Our qualitative analysis revealed how students' perceived the roles of voice and text input in shaping their problem-solving process, as well as the reported drawbacks and advantages of each modality. We discuss implications for future multimodal tools and instructional design in computing education.
K. Riegel, Y. Hua, Paul Denny et al.· Annual Conference on Innovat...· 0 citations
Generative artificial intelligence (AI) has unsettled a long-standing premise of second language (L2) writing research: that the language through which a writer’s voice is realised originates with the learner. Voice (what a text conveys about its writer) and psychological ownership (whether writers feel the text is theirs) capture different aspects of a writer’s relationship to a text, yet have been theorised separately and rarely examined together. Using a sequential explanatory mixed-methods design, this study operationalised voice and ownership together in one sample of Saudi EFL undergraduates, most in their first year and new to AI, a population rarely examined in identity-focused research on AI-assisted writing. A questionnaire was completed by 178 students; 17 were interviewed using maximum variation sampling. Data were analysed using descriptive and inferential statistics and reflexive thematic analysis. AI use was assistance-oriented rather than generative, supportive use exceeding full-text generation ( d = 0.86). Perceived voice was near the midpoint while ownership fell significantly below it; the two were strongly related ( r = .608) yet not redundant. Supportive use was the strongest predictor of both perceptions, whereas generative use predicted ownership but not voice, and proficiency made a small, tentative contribution to voice only. Interviews showed ownership to be a graded judgement turning on transformation and accountability, and voice to rest on recognisability and level-match; full-text generation occupied the authorship boundary. These findings suggest pedagogy and assessment should focus less on whether AI is present than on whether learners understand, transform, and authorise its contribution.
Aser Altalib· International Journal of TES...· 0 citations
People with physical and motor disabilities face significant barriers when interacting with image creation applications that rely on keyboard input, touchscreens, or complex graphical interfaces. Although modern AI image generation models produce high-fidelity images from textual descriptions, they remain dependent on typed input, which is inaccessible to users with limited hand mobility. To address this gap, this paper proposes a voice-to-image assistive system that enables users to create images exclusively through voice commands. The system integrates three tightly coupled modules: (i) a Speech-to-Text (STT) engine based on OpenAI Whisper, which transcribes spoken commands into raw text; (ii) an NLP-based prompt refinement module that applies tokenization, grammatical correction, and semantic augmentation to produce diffusion-friendly image prompts; and (iii) a Stable Diffusion image synthesis backend that generates high-resolution images from the refined prompts using GPU-accelerated latent diffusion. The system achieves STT accuracy of up to 100% for simple commands and 96% for long descriptive inputs, with prompt-to-image semantic alignment reaching up to 97%. Average image generation time is 6–12 seconds on GPU hardware. This work demonstrates the practical viability of multimodal AI pipelines for assistive applications and outlines directions for future development, including multilingual support, mobile deployment, and integration with existing assistive technology ecosystems.
Dasari Sri Krishna, Vaddi Radhesyam, Lakshmi Aiswarya Narikimilli et al.· 2026 4th International Confe...· 0 citations
Research on artificial intelligence (AI)-assisted rewriting activities in Systemic Functional
Linguistics–Genre-Based Approach (SFL–GBA) L2 writing remains limited. In this study, revision tasks using ChatGPT were embedded
into a 15-week SFL–GBA writing class at a university in Japan. EFL learners wrote pre-essays in the discussion genre within a time
limit and then revised the same text using ChatGPT. Their experiences were collected through a post-essay questionnaire on how
ChatGPT prompted them, how well they understood the feedback, and how they felt about using it again. Pre- essay and post-revision
drafts were explored to understand their selected SFL resources, including changes in the uses of Nominalization, Attribution, and
theme patterns. Low-proficiency first-year students tended to use ChatGPT for local substitutions, while high-proficiency
first-year students were more likely to seek guidance on word choice and genre direction. Second-year university students used
ChatGPT more selectively for sentence restructuring and phrasing refinement. While EFL learners’ evaluations of the feedback were
generally positive, their willingness to continue using ChatGPT varied across groups and seemed to be related to their perceptions
of the clarity of the feedback. Revised drafts illustrated a gradual shift toward a more academic register with higher lexical
density and more nominalizations. Changes in the use of interpersonal positioning and the use of contrasting themes to organize
counterarguments were also identified.
Akiko Nagao· Australian Review of Applied...· 0 citations
