Nemotron 3 Diarization is an open-weight speech model from NVIDIA that answers one question: who spoke when. Speech recognition gives you the words. Diarization gives you the speaker turns, including the moments when people talk over each other. Put the 2 together and you get a transcript with names on it.
NVIDIA released it on 23rd September 2026. At 100 million parameters, it handles up to 8 speakers, in live streams or finished recordings, and ranked first of 12 systems on VoiceArena's Diarization-Bench with a 14.72% error rate, against 19.3% for the next system.
| Maker | NVIDIA |
| Size | About 100 million parameters |
| Speakers | Up to 8 at once, with overlapping speech |
| Modes | Offline and streaming, from 0.32 to 30.4 seconds of buffered audio |
| Input | 16 kHz mono audio: .wav, .flac, .opus, .mp3 |
| Runs on | Linux with NVIDIA Ampere, Hopper or Blackwell GPUs, through the NeMo Speech toolkit |
| Licence | OpenMDW 1.1 |
For business people
The problem: a transcript without speakers is hard to use. You can read "I'll send the report" and "Yes, by Friday", but not who committed to what. Summaries, action items, call analytics and meeting search all depend on knowing the speaker.
What it does: it labels every stretch of audio with a speaker channel, even when 2 people talk at once. It doesn't know names. It says "speaker_2 spoke from 12.4 to 15.1 seconds". The application maps that channel to a real person, from the meeting invite, a user profile or a voice print.
Why it matters: its predecessor handled 4 speakers. 8 covers most meetings. Across 8 test conditions, NVIDIA reports errors about 40% lower on average than its previous streaming model, and the gain grows with the number of speakers.
The catch: it runs on NVIDIA hardware under Linux. On a Mac or an iPhone, the supported route is Argmax Pro SDK 3, a commercial SDK that added the model at launch. Argmax ran its own benchmark: the lowest error rate of 6 systems across 11 datasets.
Limits: more than 8 speakers, heavy noise, echo, distant microphones or very long recordings all raise the error rate. NVIDIA advises testing on your own audio before any consequential use.
For technical people
How it works
- Features. 16 kHz mono audio becomes a Mel-spectrogram at a 10 ms step, stacked 8 times into 80 ms frames.
- Encoder. A 31-layer Transformer with rotary positional embeddings (RoPE), then a Conv1D layer back to 10 ms resolution.
- Output. A
[T, 8]tensor: for each time step, the probability that each of 8 speakers is active. 2 channels can be active in the same frame, which is how overlap works. - Arrival order. Following NVIDIA's Sortformer approach, the first new voice gets channel 0, the next channel 1, and so on. Labels stay stable from chunk to chunk without re-matching speakers.
- Streaming memory. An Arrival-Order Speaker Cache keeps what the model knows about earlier speakers. A FIFO queue keeps recent context. Right context, audio just after the chunk, helps at turn changes.
Latency options
The same model runs at 4 recommended operating points. The latency is buffered audio only: add compute, network and speech recognition on top.
| Configuration | Input buffer | Use for |
|---|---|---|
| Offline style | 30.4 s | Recordings, best accuracy |
| Low latency | 1.04 s | Live captions, meeting assistants |
| Very low latency | 0.64 s | Voice agents |
| Ultra-low latency | 0.32 s | The lowest recommended setting |
Benchmarks, as reported by NVIDIA
- VoiceArena: first of 12 systems on 139 English conversations, about 22 hours, with overlap scored and no boundary tolerance. The launch results are initial and may change.
- Against the previous 4-speaker model at 1.04 s: lower error on all 8 test conditions, from 9% better on CALLHOME to 65% better on NOTSOFAR1.
- 1 exception: on 2-speaker phone calls it's slightly worse, 5.98% against 5.68%.
- Throughput: 15,113 times real time at batch size 32 in offline mode on an RTX PRO 5000, against 2,619 times for the old model.
Quick start
pip install Cython packaging
pip install 'nemo-toolkit[asr]'
from nemo.collections.asr.models import SortformerEncLabelModel
model = SortformerEncLabelModel.from_pretrained("nvidia/Nemotron-3-Diarization")
model.eval()
# Offline-style settings, in 80 ms frames
m = model.sortformer_modules
m.spkcache_len, m.fifo_len = 264, 40
m.chunk_len, m.chunk_right_context = 340, 40
m.spkcache_update_period = 300
model._check_streaming_parameters()
for segment in model.diarize(audio=["meeting.wav"], batch_size=1)[0]:
print(segment) # "0.400 2.100 speaker_0"
To get a speaker-attributed transcript, run a speech model with word timestamps, such as NVIDIA's Parakeet TDT 0.6B v3, and give each word the speaker active at its midpoint. NVIDIA's example marks words during overlap as ambiguous rather than guessing.
⚠️ WARNING: the model outputs anonymous channels, not identities. Treat speaker assignments as probabilities, and keep the uncertainty visible in anything downstream.
How it compares with my transcript skill
My own transcript skill (see The NicAI skills catalogue ) runs fully on my Mac: whisper.cpp for the words, and voice-print clustering for the speakers. It then names speakers from enrolled voice prints.
| My transcript skill | Nemotron 3 Diarization | |
|---|---|---|
| Method | Voice-print embeddings, then clustering | End-to-end model, speaker activity per frame |
| Overlapping speech | 1 speaker per segment | Several speakers per frame |
| Live streaming | No | Yes, down to 0.32 s |
| Names speakers | Yes, from enrolled voice prints | No, anonymous channels only |
| Runs on my Mac Studio | Yes | Not supported by NVIDIA. Possible through Argmax's commercial SDK |
The best of both would be Nemotron for the turns and my voice prints for the names. That needs an NVIDIA GPU or the Argmax SDK, so for now it's one to watch, not one to install. The live demo on Hugging Face runs in the browser and takes a microphone or an uploaded file, which makes it easy to test on a real call recording first.
Further Reading
