Speech Recognition

Outstanding paper on ACL 2026 https://github.com/HITsz-TMG/Lychee-FD Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLM

https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B Audio8-ASR-0.1B is a compact autoregressive ASR model whose language-model component has only 0.1B parameters. It supports multilingual speech recognition for languages including Chinese, English, French, German, Japanese, Korean, and Cantonese. We position it as one of the smallest usable performance ASR models in the LLM era. The audio encoder backbone is based on Qwen3-ASR-0.6B, with the audio adapter and projector trained as part of Audio8-ASR. The language-model backbone is based on Ref-Pretrain-Qwen-104M.

Part of the news on Inkling is that it actually handles audio as dMel https://huckiyang.github.io/blog/inkling-audio-design.html https://x.com/huckiyang/status/2077625513384841679

There are numerous modern speech-to-text models available, which can be fine-tuned or utilized in various ways. Why is Zipformer still being used?

We've just released Zipformer Tajik model (you can use it with sherpa-onnx) https://huggingface.co/alphacep/vosk-model-tg Somewhat initial one, we will work more on it.

Great part from MERL paper above https://real-tse.github.io/assets/pdf/MERL-SA-Track2.pdf VII. METRIC ATTACK Many speech separation and target speech extraction systems optimize evaluation metrics either explicitly or implicitlyduring training, including metrics such as SI-SDR and PESQ. From the perspective of Goodhart’s Law, this practice fundamentally compromises the validity of such metrics as evaluation tools: once a metric becomes an optimization target, it ceases to function as an independent measure of system quality. ..... Given the fragility of non-intrusive metrics as demonstrated by our attack and also shown in [37], [38], we suggest that the Challenge Organizers either remove DNSMOS and spksim when calculating the official ranking or replace them with alternative speech quality and speaker similarity metrics that were not attacked either advertently or inadvertently by submitted systems

FAD optimization during training https://github.com/voidful/fd-speech

Just Turkish TTS so not very applicable to wide audience but interesting design (only 200M DiT + 25 Hz VAE from VoxCPM2) https://github.com/freyavoiceai/FreyaTTS

New STT https://huggingface.co/ai-sage/GigaAM-Multilingual

Tanel keeps winning challenges https://betrac.github.io/ https://www.linkedin.com/feed/update/urn:li:activity:7482403480888975360/ the winners of the Beyond Transcription Challenge! 🏆 Lightweight track (<6B params, no tools) 1. TalTech 2. NTT-HI-CS 3. KUSLP Heavyweight track (<36B params) 1. TalTech 2. KUSLP 3. NTT-HI-CS 🧪 The Challenge Teams were given 1,100 hours of fully synthetic doctor-patient conversations with reference SOAP notes (conversations roleplayed by Gemma 3, notes generated by Kimi K2, from our Interspeech paper), a list of allowed open-weight models and datasets, and one goal: build the best end-to-end audio-to-SOAP-note system possible. 📊 The Results Systems were evaluated with an automated medical Concept F1 scorer on held-out conversations from the same distribution. All three top teams converged on the same recipe: supervised fine-tuning on the references, followed by reinforcement learning with Concept F1 as the reward. Their systems are remarkable. Crushing hallucinations: the best baselines and cascaded systems we evaluated — built from Qwen 3 and Whisper components — hallucinate on more than 20% of claims. The top competition systems brought that below 1%. ❓ But does it generalize? The obvious objection: isn't this overfitting to synthetic data? And isn't Concept F1 a very limited metric? So we tested it. During evaluation, teams also generated notes for 272 human-acted medical dialogues — not permitted for training, and with no reference SOAP notes. Across n = 19 submitted systems, we asked two questions: 1. Does synthetic performance predict real performance? Yes, almost exactly. Real-data Concept F1 tracks held-out synthetic Concept F1 with a slope of 0.89 (lightweight) and 0.94 (heavyweight) — essentially the identity line (r = 0.97–1.00). 2. Does Concept F1 predict LLM-as-a-judge quality? (judge pipeline using Gemma 4) Yes — r = 0.83–0.87. The agreement is tightest among the strongest systems and fans out below ~0.35 Concept F1, so the metric is most trustworthy exactly where it matters. The synthetic data approach looks like a genuinely promising way forward. Plenty of open questions remain — but "train on synthetic, deploy on real" held up here.