Aug 2026· International Conference on Advanced Sensing and Intelligent Systems· Vol 14309, pp. 1430918 - 1430918-8· 0 citations· 6 references
Engineering
TL;DR
An automated pipeline for transcription and summarization of video conferences held on the Jitsi Meet platform that requires no proprietary cloud API keys and is deployable on-premise via Docker Compose, making it suitable for organizations with strict data-privacy requirements.
Abstract
This paper presents an automated pipeline for transcription and summarization of video conferences held on the Jitsi Meet platform. The proposed system integrates the Jigasi SIP gateway as a transparent in-call participant that captures perspeaker RTP audio streams in real time. Each stream is transcribed using the Vosk large-vocabulary speech recognition model (vosk-model-en-us-0.22), which delivers word-level timestamps and speaker attribution without relying on cloud services. A monitoring script continuously watches the Windows Downloads folder for new WebM recordings produced by Jitsi; upon detection the file is moved to a dedicated recordings directory and queued for processing. After transcription, the resulting structured text is forwarded to a Claude-based summarization engine that extracts the agenda, key decisions, action items, and participant contributions, saving the final report as a plain-text file. Experiments on 20 real-world meeting recordings demonstrate a word error rate of 14.3% for English speech and an average end-to-end latency of 38 seconds per hour of audio on a CPU-only server. The system requires no proprietary cloud API keys and is deployable on-premise via Docker Compose, making it suitable for organizations with strict data-privacy requirements.
This paper proposes a well-rounded media consumption system that utilizes speech recognition and generative Artificial Intelligence to create a user-influenced slideshow. The system runs on a Raspberry Pi, using a microphone for input and a touchscreen for output. Spoken user requests are transcribed and categorized into one of five content classes: news, daily activities, social media trends, interest-based topics, and storytelling. These categories are then converted into prompts suitable for image generation. The system uses AI to condense unstructured speech into descriptive, image-ready content, reducing cognitive overhead and minimizing screen time. Unlike conventional browsing, this approach enables passive, voice-controlled consumption of highly relevant media. Qualitative and quantitative evaluation shows that the system reliably transcribes varied speech inputs, classifies user intent with high interpretability, and generates coherent, category-aligned visuals. Specifically, the LLM-based intent classifier achieved 90% accuracy and a Macro-F1 of 0.931 on the tested prompts, while on-device transcription operated at an average CPU utilization of 7.3% with a peak SoC temperature of 48.7°C, confirming the feasibility of the pipeline on a Raspberry Pi. The use of AI in this context enhances personalization, reduces interaction friction, and supports timeefficient engagement with digital media.
Mohammed Ghazal, Syrin Alabrach, Abdalla Gad et al.· International Conference on...· 0 citations
Caption Studio is a transparency-first speech and audio intelligence platform that transforms spoken audio and video into structured, searchable content through automated transcription, speaker diarization, speech analytics, signal-level audio analysis, and subtitle generation. The system is built on a FastAPI backend with a real-time dashboard and adopts a three-layer architecture comprising (i) a transcription and diarization core based on Whisper-class automatic speech recognition and pyannote speaker diarization, (ii) an audio intelligence layer that extracts acoustic and linguistic features, including waveforms, spectrograms, pitch, speaking rate, silence, filler-word frequency, and sentiment, directly from the audio signal, and (iii) an integration layer that supports data export and downstream workflow integration. A principal contribution of this work is the transparency-first framework, in which every reported metric is explicitly identified as measured, derived, or unavailable, thereby improving the traceability, interpretability, and reliability of speech analytics. The paper presents the system architecture, benchmarking methodology, explainability and uncertainty framework, and key considerations for enterprise-scale deployment.
C. Chin, Jianhua Zhang, M. Venkateshkumar· 0 citations
This paper describes TalTech's submissions to the Beyond Transcription Challenge (BeTraC), which requires generating SOAP notes directly from long doctor-patient conversation recordings, without intermediate transcription. After screening open-weight speech LLMs for long-audio robustness, we adapted Voxtral Mini (lightweight track) and Voxtral Small (heavyweight track) with LoRA supervised fine-tuning followed by DAPO reinforcement learning that uses the challenge metric, Open Medical Concept F1, as its reward. Our systems ranked first in both tracks, and an independent LLM-as-a-judge evaluation showed the lowest hallucination rate among all submissions, indicating that reinforcement learning against a concept-matching metric need not compromise factual reliability. We also find that fine-tuning on text transcripts transfers well to speech input and appears to improve robustness on out-of-domain real recordings.
This paper presents the tttAI system submitted to the TSA-ASR task of the SmartGlasses Challenge 2026, evaluated on both two-person dialogues (Track 1) and multi-party meetings (Track 2). The task requires time-stamped speaker-attributed speech recognition from smart-glasses recordings. This is particularly challenging due to long-form audio, multiple speakers, and frequent overlapping speech. We proposed a cascaded architecture consisting of speaker diarization, overlap detection, target-speaker extraction, post-processing, and automatic speech recognition. The diarization module extracts features via WavLM-Large, performs frame-wise speaker classification with a Conformer encoder, and then generates global speaker segments through embedding clustering. For overlapped regions, we apply a WeSep-based target-speaker extraction model with ECAPA-TDNN speaker embeddings. When the extraction is unreliable, a dominant-speaker fallback strategy is used. The final system uses FireRedASR2-AED with the first microphone channel. The submitted system has a total parameter count of approximately 1.53B. On Track 1, our system achieves a tcpCER of 7.10%. On Track 2, it achieves a tcpCER of 34.04% and ranks second on the leaderboard.
Xuanji He, Gaoyang Dong, Xiaoxiao Li et al.· 0 citations
India is home to an incredible number of languages which leads to the production of substantial amounts of news articles provided in their regional languages including Telugu, Tamil, Hindi, Bengali, Kannada, and Malayalam. Unfortunately, current systems for processing this data do so independently, rather than as part of a comprehensive framework utilizing all the necessary components in one queue of processing pipelines; OCR extraction, Translation, Summarization, and Bias Detection must all be completed one at a time, and do not allow for seamless data passing between functions. In this paper, we will present Bharat Sum, a multilingual news summarization and bias detection system that incorporates OCR capabilities into five different types of processing stages; OCR Text Extraction using Tesseract, Automatic Language Detection using LangDetect, Topic Segmented Abstractive Summarisation using mT5, Translation to English using mBART, and Sentiment based Bias Classification using DistilBERT - all accessible through a single scalable architecture running on commodity hardware and implemented via Streamlit. Our testing involved 150 news articles covering each of the five languages named above. The results achieved were as follows; OCR extraction accuracy of 89.7%, Language Detection Accuracy of 94.2%, Summarisation Quality (ROUGE-L F1) of 92.1% Translation consistency of 91.4% and Sentiment Classification Accuracy of 88.6%. The average end-to-end processing time was between 10 and 16 seconds. Our analysis of Bharat Sum has revealed that it significantly outperforms previous single function systems by providing an Integrated, Real-Time Multilingual Processing capability which currently does not exist in this context. Bharat Sum has the potential to address significant gaps in the research literature regarding the analysis of Integrated Multilingual Media, and will likely serve as an economically viable solution for organisations conducting Digital Journalism, Media Monitoring, or Accessing Multilingual Information.
Farooq Sunar Mohammad, E.Sneha, B.Kavya et al.· 2026 7th International Confe...· 0 citations
Results show that supervised fine-tuning provides the largest gain, while synthetic-speech LoRA adaptation and reinforcement learning further improve robustness, while synthetic-speech LoRA adaptation and reinforcement learning further improve robustness.
Hao Wu, Rong-Qi Han, Zhen Wang et al.· 0 citations