Whistle, a 16.9 MB speech model by Cactus

An open speech-to-text model in 16.9 MB: 7 languages, word timestamps, keyword biasing, CPU only, first token in 11 ms. Apache 2.0.

Whistle is a speech recognition model from Cactus Compute, the Y Combinator company behind tiny on-device models. It's 1 file of 16.9 MB that runs on a CPU with no dependencies, on a phone, a watch, a Raspberry Pi or a microcontroller-class board.

It transcribes English, German, French, Spanish, Italian, Dutch and Polish, and detects the language by itself. On an Apple M4 Pro CPU, the first word comes out after 11 ms for a 10-second clip.

The real trick is the pairing. Whistle loads into the same engine as Needle, Cactus's tool-calling model, so 1 binary takes a voice clip and returns a function call. Speech in, action out, all on the device.

What it does

  • Transcription. 16 kHz mono audio of any length: 1 pass up to 30 seconds, windowed beyond that.
  • Word timestamps. Every word with its start, end and probability.
  • Keyword biasing. Pass names or jargon (keywords=["Siobhan", "Krzysztof"]) and the decoder favours them. Useful for product names and people's names, which small models usually mangle.
  • Speech embeddings. The encoder output, 1 row per 80 ms, without a transcript. Useful for speaker or sound matching.
  • Silence check. A quiet clip returns an empty transcript without running the decoder, so a wake-word loop costs almost nothing.

How it compares

Cactus benchmarked Whistle against the 2 small models people actually ship on devices, on 10 seconds of audio on an M4 Pro CPU:

Model Size First token Decode speed Languages
Whistle 16.9 MB 11.1 ms 1,319 tokens/s 7
Whisper base 145.3 MB 73.2 ms 266 tokens/s 99
Moonshine tiny v2 41.9 MB 22.8 ms 262 tokens/s English

On accuracy (word error rate), Cactus reports:

  • Whistle wins on LibriSpeech clean and other, on SPGISpeech and Earnings-22 (financial calls), and on the FLEURS multilingual average.
  • Whisper base wins on TED-LIUM talks, on AMI meetings and on the MLS multilingual average, at 9 times the size.

Whisper pads every clip to 30 seconds, so its first-token time is flat. Whistle's grows with the clip: 5.9 ms at 5 seconds, 36.3 ms at 30. For short voice commands that's the difference between instant and noticeable.

Cactus checked that no test audio appears in its training data, by audio checksum and speaker ID. The accuracy figures for Whisper and Moonshine are the ones their authors published, not a re-run.

Get started

pip install cactus-compute            # add [mic] for other sample rates and microphone capture
import cactus

r = cactus.transcribe("clip.wav", word_timestamps=True, keywords=["Kaltura"], language="de")
print(r["text"])
  • Each call returns the text, the language, the time to first token and the decode speed.
  • cactus whistle playground transcribes from the microphone in the terminal.
  • cactus whistle compare runs the same clip through Whistle, Whisper and Moonshine side by side, with timings. Good for checking on your own audio before you believe any chart.
  • ⚠️ The Python package sends anonymous usage counts by default. Set CACTUS_TELEMETRY=0 to stop it. The C engine itself reads no environment variables.

Voice to tool call in 1 binary

cactus --model needle3.cact --model whistle.cact --tools tools.json --audio clip.wav
{"function_calls":[{"name":"set_lights","arguments":{"room":"kitchen","on":false}}],
 "confidence":0.94,
 "audio_text":"turn off the kitchen lights",
 "audio_language":"en"}

The engine transcribes, matches the transcript against your tools and returns 1 JSON object. Your code never handles the transcript. Pebble already runs Needle locally for its Index Ring, a ring with no screen and no guaranteed internet.

Inside the model

  • Front end: 80 log-mel bins at a 10 ms hop, then a convolutional stem that leaves 1 frame per 80 ms. 30 seconds becomes 375 frames.
  • Encoder: 8 Simple Attention blocks, the same code as Needle, attending over the whole clip at once. It isn't a streaming model.
  • Decoder: 8 layers at width 512 that read the encoder through a gated cross-attention. The audio side is computed once per clip, so 5 beams cost 5 short text caches, not 5 passes over the audio.
  • Adjustable depth: every decoder depth from 2 layers up was trained as its own model. --audio-depth trades accuracy for speed at load time.
  • Vocabulary: 8,192 text pieces plus 1 token per language, so the detected language comes out as a token.

The engine ships prebuilt for 17 targets: macOS, Linux, Windows on ARM, Android, iOS, watchOS, RISC-V, MIPS, the browser and WASI. The speech C API is 3 calls: cactus_load, cactus_transcribe and cactus_embed.

Limits

  • The comparison set is small models only. Whistle beats Whisper base, not Whisper large or Parakeet. For accuracy on long, messy audio, a bigger model still wins.
  • Meetings are its weak spot. Whisper base, itself a small model, beats it on AMI meeting audio.
  • Not streaming. It transcribes clips. Live captions need you to chunk the audio yourself.
  • 7 languages, all European. No Asian languages, no Portuguese.
  • Vendor numbers. I found no independent benchmark yet. 13 fine-tunes on Hugging Face suggest people are testing it.

Value

My dictation app, Dictee, runs Parakeet TDT 0.6B v3 on MLX, about 2.5 GB (see Dictee: my own local dictation app ). Whistle won't replace it on the Mac: accuracy matters more than size there, and Parakeet is in another class.

Where Whistle fits is everywhere a 2.5 GB model can't go. A voice command on a phone or a watch, a Home Assistant satellite that works offline, a Raspberry Pi in the studio that listens for "start recording" and calls a tool. It speaks my 3 languages, English, French and German, and the Needle pairing turns speech into an action with no cloud in between.

For a slightly larger, English-only model with near-Parakeet accuracy, see Phonon-2. For Cactus's 22 MB vision model on the same engine, see Mirage, a 22 MB vision model by Cactus.

Apache 2.0 for the weights, the engine and the prebuilt binaries.

Further reading

The weights and the guides:

How to fine-tune Whistle on your own vocabulary:

The engine source:

NicAI
Written by NicAI, Nic's AI assistant, for his personal knowledge base. Researched and drafted by the model, not hand-written by Nic. Verify anything you plan to act on.