NVIDIA's Nemotron 3.5 Transcribes an Hour of Audio and Translates It Too
Nemotron 3.5 Transcribe handles transcription, speaker diarization, translation, and question answering in a single model. NVIDIA also released the Granary v2 dataset and training tools so developers can fine-tune it for specialized vocabulary.
On October 28, NVIDIA released Nemotron 3.5 Transcribe, a model that turns speech into text. Alongside it, the company published Granary v2, a multilingual training dataset. The model transcribes 40 languages. Of those, 31 can be translated into English, or from English. Speaker diarization, which works out who said what, is available in 13 languages.
The central idea is converting a recording into text shaped for whatever you need. The model handles recordings up to an hour long and can distinguish up to eight speakers. Using a prompt, the instruction you give an AI, you can ask for speaker labels or request an English translation. You can also simply ask what was decided in a meeting.
How far can one recording take you
Give the same model different instructions and you get text in the form you need. For a medical recording, you can supply drug names or test names likely to come up in advance to help recognition. There is also a feature that timestamps each individual word. Agora, a company that builds real-time voice and video platforms, demonstrated using this to display a transcript alongside the playback screen of a recorded class.
Language support varies by use case. The question and answer feature, which lets you ask about the contents of a recording, was trained in English. The model is built to process already recorded files rather than live streams. NVIDIA advises checking names, the points where speakers change, and answers to questions against the original recording.
How accurate is the transcription
NVIDIA says the model ranked first among open-weight models on Hugging Face’s English speech recognition benchmark. Its word error rate, the share of words transcribed incorrectly, averaged 5.33%. The lower the number, the fewer words it got wrong. In a separate evaluation by Artificial Analysis, it tied for first among models in the same category, with an error rate of 2.8%.
Image: huggingface.co
The company also highlighted results on recordings with multiple speakers. According to NVIDIA, in David AI’s 21-language evaluation, it had the lowest diarization error rate among the 14 systems tested. On VoiceArena, which evaluates full English meetings, it also placed first among 14 systems. In throughput testing, NVIDIA said a single GPU processed 100 hours of audio in under three minutes. That result came from running large batches of 30-second clips on a lightweight version due for release.
Can it be tuned to specialized terminology
Developers can download the model from Hugging Face and ModelScope and run it in the environment of their choice. Granary v2, released alongside it, catalogs 463,062 hours of data across 48 languages. That is a wider range than the 25 languages in the previous version, Granary v1. The source audio has to be downloaded separately, in line with each dataset’s licensing terms.
Image: huggingface.co
To match the speech patterns and notation conventions of a specialized field, you can run additional training with the NeMo Speech toolkit. NVIDIA said that in a medical speech case, evaluated on speakers not used in training, the word error rate dropped 45.8% compared with the best-performing prompt. This release widens the path from simply turning recordings into text toward tailoring terminology and output format to the work at hand. The open question is whether that performance holds up in the user’s own language and recording conditions.