This study proposes a multi-stage intelligent documentation framework that integrates a 12-layer Conformer architecture, BERT-BiLSTM-CRF semantic modeling, and BART-based structured text generation that illustrates the effectiveness of deep time-frequency feature extraction and contextual sequence modeling for complex noisy signals.
Abstract
Accurate clinical speech recognition remains challenging because rapid pronunciation, domain-specific terminology, and background noise often degrade automatic speech recognition and subsequent medical record generation. This study proposes a multi-stage intelligent documentation framework that integrates a 12-layer Conformer architecture, BERT-BiLSTM-CRF semantic modeling, and BART-based structured text generation. The Conformer encoder captures both local acoustic characteristics and long-range contextual dependencies, while the semantic module performs medical entity recognition and normalization to enhance terminology consistency. The extracted information is subsequently incorporated into a BART generator with clinical knowledge prompts to produce standardized SOAP-compliant medical records. Experimental results demonstrate a word error rate of 6.3%, medical term accuracy of 95.8%, low response latency of approximately 940–960 ms, and generation quality approaching physician-written records. Beyond clinical documentation, the proposed framework illustrates the effectiveness of deep time-frequency feature extraction and contextual sequence modeling for complex noisy signals, offering methodological insights for electromagnetic signal interpretation, antenna measurement data processing, and intelligent information extraction in propagation-related applications.
A prompt-guided and quality-aware text–audio intent recognition framework that can improve the robustness of intent recognition under challenging acoustic conditions in elderly-oriented interaction scenarios is proposed.
Zhi-Min Wei, Shu-Hao Tian, Yan-Zhen Wang et al.· Italian National Conference...· 0 citations
The hybrid pipeline integrating rule-based dictionary normalization with large language model (LLM)–based postprocessing significantly improved Korean-English code-switched medical ASR accuracy.
Chanryeong Oh, Yul Hwangbo, Wonjoong Cheon et al.· Journal of Medical Internet...· 0 citations
This paper investigates how multilingual medical adaptation reshapes the internal representations of Whisper models through layer-wise encoder analysis, and shows that English medical fine-tuning produces the dominant encoder shift, whereas multilingual continuation largely preserves the adapted representation space.
Souranil Kahali, Rituparna Bose, Abner Hernandez et al.· 0 citations
Results show consistent gains from lightweight domain adaptation on both held-out synthetic data and real-world recordings, confirming that synthetic data generation combined with LoRA-based fine-tuning is an effective and computationally practical strategy for improving ASR accuracy in specialized technical domains where labeled speech is scarce.
Aman Kumar, Ahmed K. Farahat, Huimin Zhuge et al.· PHM Society European Confere...· 0 citations
Cognitive impairment (CI) is a growing public health concern. Early and accurate diagnosis is critical for enabling timely intervention and improving patient outcomes. Speech-based CI detection has emerged as a promising non-invasive approach, as speech signals encode both linguistic and acoustic markers associated with cognitive decline. Recent advances in large language models (LLMs) further strengthen the potential of speech-based assessment by enabling more expressive representation learning and improved generalization across diverse speakers, recording devices, and clinical environments. Moreover, multimodal learning by jointly modeling linguistic and acoustic features allows for a more comprehensive characterization of cognitive and behavioral changes related to CI, leading to more reliable detection. In this work, we propose a multimodal CI detection framework based on open-source LLMs that integrates speech audio and corresponding transcripts while preserving patient privacy. Acoustic embeddings are extracted directly from speech signals, while textual embeddings are generated from automatically transcribed speech. These modality-specific embeddings are then concatenated to create a combined feature vector and used for downstream classification, without requiring access to raw or sensitive patient data. The proposed approach is evaluated on the ADReSS20 and ADReSSo21 benchmark datasets. Experimental results show that the proposed multimodal framework achieves an CI classification accuracy of 92.4% and consistently outperforms single-modality baselines. Our work establishes a new state-of-the-art for CI identification, with the proposed method demonstrating superior cross-dataset generalization. This advance highlights the power of an LLM-based multimodal framework that fuses linguistic and acoustic data to enable robust, scalable, and non-invasive screening.
Ying-lei Huang, Xin Wang, Yuhan Su et al.· 0 citations
Developing Automatic Speech Recognition (ASR) systems for specialized medical domains is challenging in lowresource settings due to the limited availability of annotated speech data. Synthetic speech generated using text to speech (TTS) systems is often used to augment training data, but directly mixing synthetic and real speech can introduce distribution mismatch that degrades encoder representations. This work introduces Real-Governed Representation-Calibrated Training (RG-RCT), a training strategy that regulates the influence of synthetic speech at the representation level. The proposed framework combines reliability-based confidence estimation, representation alignment, and layer-wise representation governance to stabilize the encoder feature space while preserving acoustic diversity from multi-speaker synthetic data. Experiments on medical-domain speech datasets using Wav2Vec2 and Whisper demonstrate that RG-RCT consistently outperforms conventional strategies such as direct mixing and confidence-aware training. The proposed method achieves the lowest Word Error Rate (WER), reaching 17.1% and 16.6% for Telugu and 23.3% and 25.6% for Kannada. These results indicate that regulating encoder representation geometry enables more effective utilization of synthetic speech for low-resource medical ASR.
S. Devarakonda, Ravi Sastry Kolluru, Srihari Bandarupalli et al.· International Conference on...· 0 citations
