226 lines
8.9 KiB
Markdown
226 lines
8.9 KiB
Markdown
<div align="center">
|
||
|
||
# Whisper GGML
|
||
|
||
_On-device speech-to-text for Flutter, powered by [whisper.cpp](https://github.com/ggml-org/whisper.cpp) v1.9.1._
|
||
|
||
<p align="center">
|
||
<a href="https://pub.dev/packages/whisper_ggml">
|
||
<img src="https://img.shields.io/pub/v/whisper_ggml?logo=dart&color=blue" alt="pub">
|
||
</a>
|
||
<a href="https://github.com/ggml-org/whisper.cpp">
|
||
<img src="https://img.shields.io/badge/whisper.cpp-v1.9.1-green" alt="whisper.cpp">
|
||
</a>
|
||
<a href="https://buymeacoffee.com/sk3llo" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="21" width="114"></a>
|
||
</p>
|
||
|
||
Transcribe audio files or **transcribe live while the user speaks** — fully
|
||
on-device, no server, no API keys.
|
||
|
||
</div>
|
||
|
||
## Highlights
|
||
|
||
- 🎙 **Live transcription** — partial transcripts stream in while recording,
|
||
refined as more audio arrives. The model loads once per session and an
|
||
adaptive energy gate keeps silence from producing hallucinated text.
|
||
- 📄 **File transcription** — one call to transcribe a recording.
|
||
- 📦 **Offline-first** — models download once and are cached, or ship them
|
||
in your app's assets for fully offline use (see the example app).
|
||
- 🌍 **99 languages** — pick one (`'en'`, `'fr'`, `'de'`, …) or use
|
||
`'auto'` to detect.
|
||
- 🎛 **Decoding controls** — vocabulary biasing, context conditioning, and
|
||
non-speech token suppression exposed from whisper.cpp.
|
||
- ⚡ **Fast** — whisper.cpp v1.9.1 with Accelerate on Apple platforms;
|
||
an 11-second clip transcribes in ~0.4 s with the `base` model on an
|
||
Apple Silicon Mac.
|
||
|
||
## Supported platforms
|
||
|
||
| Platform | Minimum version |
|
||
|----------|-----------------|
|
||
| Android | API 21 |
|
||
| iOS | 15.6 |
|
||
| macOS | 10.15 |
|
||
| Windows | 10 (x64) |
|
||
| Linux | x64 |
|
||
|
||
## Installation
|
||
|
||
```yaml
|
||
dependencies:
|
||
whisper_ggml: ^2.6.0
|
||
```
|
||
|
||
Requires Dart 3.7+ (Flutter 3.29+).
|
||
|
||
## Quick start
|
||
|
||
```dart
|
||
import 'package:whisper_ggml/whisper_ggml.dart';
|
||
|
||
final controller = WhisperController();
|
||
|
||
final result = await controller.transcribe(
|
||
model: WhisperModel.tiny,
|
||
audioPath: '/path/to/audio.wav',
|
||
lang: 'en',
|
||
);
|
||
|
||
print(result?.transcription.text);
|
||
```
|
||
|
||
Pass `onProgress: (percent) => ...` to receive transcription progress
|
||
as a 0–100 percentage (coarse steps) while inference runs.
|
||
|
||
Pass `withSegments: true` to also get per-segment timestamps in
|
||
`result.transcription.segments` (each segment has `fromTs`/`toTs`
|
||
`Duration`s and its `text`); add `splitOnWord: true` for one segment
|
||
per word instead of per phrase.
|
||
|
||
### Speaker-turn detection (diarization)
|
||
|
||
With the tinydiarize model (`WhisperModel.smallEnTdrz`, English only),
|
||
`diarize: true` marks the segments after which the speaker changes:
|
||
|
||
```dart
|
||
final result = await controller.transcribe(
|
||
model: WhisperModel.smallEnTdrz,
|
||
audioPath: '/path/to/audio.wav',
|
||
lang: 'en',
|
||
diarize: true,
|
||
withSegments: true,
|
||
);
|
||
|
||
for (final segment in result?.transcription.segments ?? []) {
|
||
print('${segment.text}${segment.speakerTurnNext ? ' [speaker turn]' : ''}');
|
||
}
|
||
```
|
||
|
||
This detects turn boundaries; it does not label or count speakers. With
|
||
regular models `diarize` has no effect.
|
||
|
||
The model is downloaded automatically on first use. Non-WAV input is
|
||
converted with the bundled FFmpeg — except on Windows and Linux, where
|
||
FFmpeg is not bundled: an `ffmpeg` executable on `PATH` is used when
|
||
present (on Linux, `apt install ffmpeg` or equivalent), otherwise the
|
||
input must already be a 16 kHz mono WAV (the format the `record` package
|
||
produces with `AudioEncoder.wav`, `sampleRate: 16000`, `numChannels: 1`).
|
||
|
||
## Live (streaming) transcription
|
||
|
||
`transcribeLive` accepts any stream of **16 kHz mono little-endian PCM16**
|
||
audio and emits progressively refined transcripts while the audio flows.
|
||
With the [`record`](https://pub.dev/packages/record) package:
|
||
|
||
```dart
|
||
final pcmStream = await recorder.startStream(const RecordConfig(
|
||
encoder: AudioEncoder.pcm16bits,
|
||
sampleRate: 16000,
|
||
numChannels: 1,
|
||
));
|
||
|
||
final session = await controller.transcribeLive(
|
||
model: WhisperModel.base,
|
||
pcm16Stream: pcmStream,
|
||
lang: 'en',
|
||
);
|
||
|
||
session.partials.listen((text) {
|
||
print(text); // full transcript so far, not a delta
|
||
});
|
||
|
||
// Later — stop recording, finalize, and free the model:
|
||
await recorder.stop();
|
||
final finalText = await session.stop();
|
||
```
|
||
|
||
Good to know:
|
||
|
||
- The model stays loaded for the whole session; inference runs on a
|
||
background isolate and never blocks the UI.
|
||
- An **adaptive energy gate** keeps silence away from the decoder, which
|
||
otherwise hallucinates on silent audio. For unusually loud rooms or quiet
|
||
speakers, tune `gateNoiseFloorCap`, `gateVoiceRatio`, and `gateRmsMin`.
|
||
- Only one live session can run at a time.
|
||
- Real non-speech sounds (knocks, clicks) may transcribe as bracketed
|
||
annotations like `[door slams]`.
|
||
- **Bring your own model:** pass `modelPath:` instead of `model:` to run
|
||
the session against any local ggml model file. For full manual control,
|
||
`startWhisperLiveSession(modelPath: ...)` is exported too — no audio
|
||
stream wiring, you call `session.feed(pcm16Bytes)` yourself.
|
||
- **Session-per-utterance?** Pass `keepModelLoaded: true` and the model is
|
||
parked in native memory when the session stops, so the next session (or
|
||
one-shot `transcribe`) with the same model starts without the
|
||
multi-second load. See "Keeping the model loaded between transcriptions"
|
||
below — the same cache, memory trade-off, and `releaseModel()` apply.
|
||
|
||
## Models
|
||
|
||
| Model | Multilingual | English-only |
|
||
|-------|--------------|--------------|
|
||
| tiny | `WhisperModel.tiny` | `WhisperModel.tinyEn` |
|
||
| base | `WhisperModel.base` | `WhisperModel.baseEn` |
|
||
| small | `WhisperModel.small` | `WhisperModel.smallEn` |
|
||
| medium | `WhisperModel.medium` | `WhisperModel.mediumEn` |
|
||
| large-v3 | `WhisperModel.large` | — |
|
||
|
||
Smaller models are faster; larger models are more accurate. `tiny` and
|
||
`base` are good defaults for live transcription; `small` is a strong
|
||
accuracy/speed balance for file transcription on modern phones.
|
||
|
||
## Decoding options
|
||
|
||
Available on both `transcribe` and `transcribeLive`:
|
||
|
||
| Option | Default | What it does |
|
||
|--------|---------|--------------|
|
||
| `initialPrompt` | `null` | Biases decoding toward the vocabulary, names, and punctuation it contains — useful for domain-specific terms that otherwise get misrecognised. Decoding also mimics the prompt's *style*: an unpunctuated prompt tends to produce unpunctuated output. |
|
||
| `noContext` | `false` | Stops whisper from conditioning on prior-segment transcripts (like Python whisper's `condition_on_previous_text=False`). Helps against hallucinated repetition on short, independent utterances. |
|
||
| `suppressNonSpeechTokens` | `false` | Suppresses bracketed annotations such as `[BLANK_AUDIO]` or `[music]`. Side effect: real sounds may decode as plausible-looking words instead, which is why the example keeps it off. |
|
||
|
||
## Keeping the model loaded between transcriptions
|
||
|
||
By default every `transcribe` call loads the model from disk (seconds for
|
||
the small models and up) and frees it when the request completes. For
|
||
push-to-talk dictation, voice notes, and other short repeated requests
|
||
against the same model, that load dominates the per-utterance latency.
|
||
Pass `keepModelLoaded: true` to park the loaded model in native memory
|
||
instead — the next transcription with the same model skips the load:
|
||
|
||
```dart
|
||
final result = await controller.transcribe(
|
||
model: WhisperModel.base,
|
||
audioPath: audioPath,
|
||
keepModelLoaded: true, // model stays resident for the next request
|
||
);
|
||
|
||
// When dictation is over, free the parked model:
|
||
await controller.releaseModel();
|
||
```
|
||
|
||
Good to know:
|
||
|
||
- A reused model transcribes **exactly like a freshly loaded one** — no
|
||
text or decoder state carries over between requests.
|
||
- The parked model keeps its weights in RAM (from ~100 MB for `tiny` up to
|
||
several GB for the large models) until you release it. On phones,
|
||
release it when dictation ends rather than keeping it forever.
|
||
- One model stays resident per process; parking a different model
|
||
replaces and frees the previous one. A request for a *different* model
|
||
with `keepModelLoaded: false` leaves the parked one alone.
|
||
- Requests stay fully concurrent: a second transcription that arrives
|
||
while the parked model is in use simply loads its own copy, exactly as
|
||
before.
|
||
|
||
- The native engine is compiled with `-O3` on all platforms, including
|
||
debug builds on iOS/macOS, Windows (`/O2`), and Linux — transcription
|
||
speed there is close to release.
|
||
- Windows and Linux x64 builds target **AVX2** by default, like upstream
|
||
whisper.cpp's standard x64 binaries (supported by virtually every x64 CPU
|
||
since ~2013). For very old CPUs, build with `-DWHISPER_GGML_AVX2=OFF`.
|
||
- Android debug builds run the Dart layer in JIT mode; use `--release` for
|
||
representative performance.
|
||
- The bundled whisper.cpp v1.9.1 is roughly **15× faster** than the engine
|
||
in versions before 2.0.0.
|