250 calls. One small box.
Measured on 4 vCPU / 4 GB with fake providers. Pipecat 1.12.0 held 1 at the same latency limit.
Every square is one concurrent call. Median of 3 runs with p95 voice-to-voice at most 1110 ms; both systems ran their own provider clients against fake providers over TLS, with Silero VAD and Smart Turn on every call. How it was measured
What is Brisvox?
Brisvox is an open-source Rust engine for real-time voice agents. It runs phone calls and browser sessions where a caller talks, the agent listens, thinks with an LLM and its tools, and answers, and the caller can interrupt at any moment.
One process serves hundreds of isolated calls per container, with the STT, LLM and TTS providers you already use. MIT or Apache-2.0, self-hosted.
Flat until the CPU runs out.
Every call is an isolated async session, and model inference runs on its own pool. Latency stays at the providers' floor until the CPU is full; twice the container gave about 1.9 times the calls.
Latency as the load grows
Voice-to-voice per load step, 8 vCPU / 8 GB container, one run per system (the median run).
- Brisvox, p95
- Brisvox, p50
- Pipecat 1.12.0, p95
- SLO, 1110 ms
Brisvox: at the providers' floor up to about 480 calls
780 calls: no failed call, no lost turn,
p95 3.7 s at that load
Pipecat: turns lost at 20 calls
in this run, and in two of three
Concurrent calls
Cost of one call
At the maximum load of the 4 vCPU / 4 GB container, idle baseline included.
Server memory at that maximum: 753 MB for 430 calls, against 2.4 GB for 10 with Pipecat 1.12.0, which loads the VAD and turn models in each of its worker processes.
All numbers and the methodThe engine adds microseconds. The rest is your providers.
Voice-to-voice latency runs from the end of the caller's speech to the first sound of the answer. Brisvox keeps its own share under a tenth of a millisecond per stage and keeps work off the critical path.
One turn, on a true time axis
From the caller's last word to the agent's first sound: 1009.6 ms at p50, of which the configured wait and the two mock providers take 950 ms.
Caller
End-of-turn decision VAD stop and STT final 450 ms
LLM first token mock provider, fixed 350 ms 358.4 ms
TTS first audio mock provider, fixed 150 ms 158.4 ms
Agent first sound at the client
| The engine's own time | p50 | p95 | Budget, p95 |
|---|---|---|---|
| End of turn to LLM request | 0.051 ms | 0.089 ms | 3 ms |
| First TTS audio to transport | 0.035 ms | 0.064 ms | 2 ms |
| Barge-in decision to Clear applied | 0.012 ms | 0.022 ms | 1 ms |
On real providers
Voice-to-voice, median to p95, 5 concurrent calls, 100 turns per system.
An interruption leaves nothing behind.
The LLM has always written more than the caller heard. Brisvox cuts the context at the heard word, so the next turn is built on the conversation that happened.
Agent, as the LLM wrote it
Your order shipped on Tuesday and should arrive by Friday. Would you like the tracking number sent to your phone?
Caller, talking over it
Wait, which address is it going to?
What the LLM context keeps
Your order shipped on Tuesday and should arrive by
- A new epoch starts. Every user turn is numbered. Bot-side frames carry the epoch they were made for, and every stage drops frames from a stale one.
- Clear goes straight to the transport. Queued audio is dropped and the client is told to flush what it buffered. Measured: 0.022 ms at p95, against a budget of 1 ms.
- The LLM round, the segmenter and the open TTS contexts are cancelled. Side-effect tools are the exception: they complete, and the result is reported on the next turn.
- The context is cut to what was played. Playback marks, playback positions and TTS word timestamps locate the last heard word. The engine waits 250 ms by default for those reports to settle.
One process. One chain of stages per call.
Calls share nothing but the inference pool, so a provider failure in one cannot touch another. Bring the STT, LLM and TTS providers you already use, over a phone line, WebRTC or a websocket.
- Transport in
AudioInVadWorkerSttStageTurnDriverLlmRunnerSegmenterStageTtsStageTransportOut
- Speech to text
- Deepgram, AssemblyAI, Gladia, sherpa-onnx (CPU), any server speaking the OpenAI Realtime transcription protocol
- Language model
- OpenAI, Anthropic, Gemini, Groq, Together
- Text to speech
- Cartesia, ElevenLabs, Deepgram Aura, Kokoro-FastAPI, Chatterbox
- Speech to speech
- OpenAI Realtime, Gemini Live
How a call works, stage by stage, with every provider and transport
Try it without any API keys
A voice agent that runs entirely on your computer: speech recognition, language model and voice, on the CPU. It needs Docker, 4 or more cores and 8 GB of memory.
git clone https://github.com/bilal07karadeniz/brisvox && cd brisvox
docker compose -f deploy/local/compose.yaml up
# then open http://localhost:8088/demo?force=ws One version, every registry
The Rust crates, the Python package and the npm packages are released together under one version number. The
short name brisvox is a real package on all three registries.
Latest release: v0.1.3
- crates.io
brisvoxServer, tools API, provider adapters and core types in one dependency - crates.io
brisvox-serverThe server binary and library - PyPI
brisvoxPython agent SDK (installs brisvox-agent) - npm
brisvoxTypeScript agent SDK and the browser client - npm
@brisvox/agentTypeScript agent SDK alone - npm
@brisvox/webBrowser module for a talk button - GHCR
brisvox-serverMulti-architecture container image
Written so an AI coding agent can pick it up cold
Claude Code, Codex, Cursor and similar tools can build on Brisvox with no prior knowledge of it.
- llms.txt and llms-full.txt
- The map and the whole user guide in one file, generated from the docs and checked for staleness by the quality gate.
- An Agent Skill
- The workflow and the common traps in two pages. Copy the folder into your project's skills directory.
brisvox-server check- Validates an agent file offline with the real loader, so an agent finds its own mistakes before anything runs.
- A JSON Schema
- The schema of the agent file, for editors and tools. The server prints it with
--print-schema.
Common questions
Short answers with their conditions. The full list is on the FAQ page.
What is Brisvox?
Brisvox is an open-source real-time voice agent engine written in Rust. One brisvox-server process runs many isolated phone or browser calls: the caller talks, speech-to-text listens, an LLM with tools answers, text-to-speech speaks, and the caller can interrupt at any moment.
The agent is one TOML or YAML file. Your own tools run in your application through the Python or TypeScript SDK, or in Rust inside the server binary.
What license is Brisvox under, and what does it cost?
Brisvox is dual-licensed under MIT or Apache-2.0, at your option. The software is free and there is no hosted cloud, so you pay only the STT, LLM, TTS and telephony providers you choose.
Each call reports what providers bill by (LLM input, cached and output tokens, STT audio seconds, TTS characters) as a session.usage event and as Prometheus totals per provider and model. The speech test fixtures in the repository are under CC BY-SA 4.0 rather than the code license.
Does Brisvox have a hosted cloud?
No. Brisvox is self-hosted only: a prebuilt binary, a container image or a build from source, on your own machines.
How many concurrent calls can one server handle?
In the cloud capacity runs of 16 and 17 October 2026 (Linux containers on AMD EPYC 9655P), one 4 vCPU / 4 GB container held 250 concurrent calls within a p95 voice-to-voice limit of 1110 ms, and 430 with no failed call or lost turn at any latency. An 8 vCPU / 8 GB container held 470 within the same limit and 780 at any latency (p95 about 3.7 s at 780).
These runs used fake providers that speak the real Deepgram and OpenAI-compatible protocols over TLS with fixed latencies, and the real Silero VAD and Smart Turn models on every call. They measure the engine and its provider clients, not the providers. At the 4 vCPU maximum Brisvox used about 1.8 MB of memory and 0.007 cores per call.
How does Brisvox handle interruptions (barge-in)?
Every user turn is a numbered epoch, and frames from a stale epoch are dropped at every stage, so audio from an answer the caller already cut off does not play. A Clear goes straight to the transport, and the LLM round, the segmenter and the open TTS contexts are cancelled.
After the barge-in the LLM context holds only the text that was actually played, so the model does not believe it said words the caller never heard.
Which languages does Brisvox support? Does it support Turkish?
English and Turkish are supported. Turkish is a first-class language: the TTS stage rewrites written forms such as amounts, dates and times into the words a person says, and the interruption policy has Turkish backchannel and command lists. An agent can list several languages and follow the caller from one to the other in the middle of a call.
Does Brisvox need a LiveKit or other media server?
No. The generic WebSocket, Twilio Media Streams and Telnyx media streaming transports are built in, and WebRTC and a SIP trunk listener are optional cargo features. Callers connect to brisvox-server directly.
Read the numbers, then run them yourself.
Every figure on this site comes from a result file in the repository, and each file starts with the command that produced it.
Last updated: