PiVo Studio: local speech from one Rust binary

PiVo is our 1.4-million-parameter text-to-speech model. Its reference runtime needs Python, NumPy and ONNX Runtime, which is fine on a workstation and awkward almost everywhere else. PiVo Studio needs none of them: it is one Rust binary that runs PiVo with its own code, from the command line, as an HTTP server or as a web studio in your browser.

The code is on GitHub and the model on Hugging Face. You can hear both voices on the PiVo page.

Try it

You need a Rust toolchain. In a clone of the repository:

cargo build --release
git clone https://huggingface.co/empero-ai/PiVo-1.4M models/PiVo-1.4M
./target/release/pivo-studio studio

The studio opens at http://127.0.0.1:8000. Type some text, pick Pi or Vo and press Speak, or Ctrl+Enter. Playback starts while the rest of the text is still being generated, with a live waveform and readouts for the time to first audio and the real-time factor. Every clip stays in a history you can replay and download as WAV.

A pronunciation inspector shows how PiVo reads your text: the normalized words, the phones for each word and any word that is missing from the dictionary. The page works offline, with no CDN and no tracking.

No runtime underneath

PiVo Studio does not wrap ONNX Runtime. It reads the four ONNX graphs with a small protobuf decoder of its own, takes the weights out of them and runs the model with hand-written f32 kernels, built on dot products and matrix-vector products that the compiler vectorizes for AVX2 and NEON. The pronunciation dictionary is compiled into the binary, so the model files are the only thing it needs at runtime, and it checks their hashes when it loads them.

We compared it with the Python reference stage by stage. The largest difference anywhere in the pipeline is 5.4 × 10⁻⁵, in the final audio samples.

Fast enough to stream

On one x86-64 CPU thread, PiVo Studio generates a sentence in 0.44 to 0.64 times its length, 1.6 to 2.3 times faster than real time. The Python reference runs at about real time. When streaming, the first audio arrives after about 30 to 50 ms, even for long documents, because the engine splits text into clauses and produces audio 10 ms at a time. The whole process peaks at about 50 MB of memory, of which the weights are about 6 MB.

We have not measured it on a phone yet. That comes next.

An API you already know

pivo-studio serve starts the same engine as an HTTP server with an OpenAI-compatible /v1/audio/speech endpoint, so existing code only needs a different base URL:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="not-needed")
client.audio.speech.create(model="pivo", voice="Vo", input="Hello from PiVo.").write_to_file("vo.wav")

Besides WAV and raw PCM it can stream: stream-pcm sends each 10 ms frame as soon as it exists, and sse sends the frames as server-sent events. There are also endpoints for the list of voices, for phonemizing text and for health checks. An API key, a limit on concurrent jobs, the host and the port are all settings.

In your own app

The engine also builds as a static or shared library with a C interface, with examples for Swift on iOS, Kotlin on Android and plain C. pivo-studio pack turns a model into a single .pivomodel file of about 6 MB to ship inside an app:

PivoEngine *e = pivo_load_bundle(bundle_bytes, bundle_len);
PivoStream *s = pivo_stream_new(e, "Hello, world.", NULL);
while (pivo_stream_next_frame(s, frame, frame_n) > 0) { /* play the frame */ }

Nothing in the engine is fixed to PiVo-1.4M. Widths, layers, heads, voices and the vocabulary come from the model files, so a future PiVo with other dimensions or more voices should load without code changes.

License

PiVo Studio is released under the Empero PiVo Attribution License 1.0. You can use, change and redistribute it, as long as anything built with it says "Made with Empero PiVo" or "Powered by Empero PiVo" where its users can see it. Commercial products also credit Empero with a link to empero.org, and companies with more than US$20M in annual revenue need a commercial license, which you can ask for at [email protected]. The PiVo weights themselves are Apache-2.0.

If you build something with it, we would like to hear about it.