250 calls. One small box.

Measured on 4 vCPU / 4 GB with fake providers. Pipecat 1.12.0 held 1 at the same latency limit.

Every square is one concurrent call. Median of 3 runs with p95 voice-to-voice at most 1110 ms; both systems ran their own provider clients against fake providers over TLS, with Silero VAD and Smart Turn on every call. How it was measured

What is Brisvox?

Brisvox is an open-source Rust engine for real-time voice agents. It runs phone calls and browser sessions where a caller talks, the agent listens, thinks with an LLM and its tools, and answers, and the caller can interrupt at any moment.

One process serves hundreds of isolated calls per container, with the STT, LLM and TTS providers you already use. MIT or Apache-2.0, self-hosted.

Flat until the CPU runs out.

Every call is an isolated async session, and model inference runs on its own pool. Latency stays at the providers' floor until the CPU is full; twice the container gave about 1.9 times the calls.

Latency as the load grows

Voice-to-voice per load step, 8 vCPU / 8 GB container, one run per system (the median run).

  • Brisvox, p95
  • Brisvox, p50
  • Pipecat 1.12.0, p95
  • SLO, 1110 ms

Brisvox: at the providers' floor up to about 480 calls

780 calls: no failed call, no lost turn,
p95 3.7 s at that load

Pipecat: turns lost at 20 calls
in this run, and in two of three

Concurrent calls

Brisvox stays at the providers' floor, about 860 ms, up to about 480 calls, and holds 780 with no failed call and no lost turn (p95 about 3.7 s at that load). Past its maximum the p95 stays near 3.7 s: turns whose Smart Turn verdict comes too late end on the 3 s fallback silence. In production, admission control refuses new calls before that point; it was off for this ramp. Files: Brisvox run 2, Pipecat run 1

Cost of one call

At the maximum load of the 4 vCPU / 4 GB container, idle baseline included.

Server memory at that maximum: 753 MB for 430 calls, against 2.4 GB for 10 with Pipecat 1.12.0, which loads the VAD and turn models in each of its worker processes.

All numbers and the method

The engine adds microseconds. The rest is your providers.

Voice-to-voice latency runs from the end of the caller's speech to the first sound of the answer. Brisvox keeps its own share under a tenth of a millisecond per stage and keeps work off the critical path.

One turn, on a true time axis

From the caller's last word to the agent's first sound: 1009.6 ms at p50, of which the configured wait and the two mock providers take 950 ms.

The amber marks are the engine's own hops between stages. At this scale each is a few thousandths of a pixel wide; they are drawn 2 px wide so they can be seen. The 43 ms after the TTS answers are the websocket transport, the 20 ms frame grid and timer overshoot. Stage medians, so they do not add up exactly. Windows run over mock providers: bench/results/v2v-sim.md
The engine's own timep50p95Budget, p95
End of turn to LLM request 0.051 ms 0.089 ms 3 ms
First TTS audio to transport 0.035 ms 0.064 ms 2 ms
Barge-in decision to Clear applied 0.012 ms 0.022 ms 1 ms
60 latency turns and 30 barge-ins over mock providers, Windows run. v2v-sim.md

On real providers

Voice-to-voice, median to p95, 5 concurrent calls, 100 turns per system.

Same agent file and providers on both: Deepgram nova-3 (STT), OpenRouter openai/gpt-6-luna with reasoning effort none pinned to one upstream (LLM), Deepgram Aura aura-2-thalia-en (TTS). Windows development machine over the public internet, one evening, so absolute numbers carry noise. The medians are within the LLM provider's own variance; the tail differs because Pipecat sent more than one LLM request in 56 of 100 turns (cause not investigated). Details and caveats

An interruption leaves nothing behind.

The LLM has always written more than the caller heard. Brisvox cuts the context at the heard word, so the next turn is built on the conversation that happened.

Turn epoch 8

Agent, as the LLM wrote it

Your order shipped on Tuesday and should arrive by Friday. Would you like the tracking number sent to your phone?

Caller, talking over it

Wait, which address is it going to?

What the LLM context keeps

Your order shipped on Tuesday and should arrive by

  1. A new epoch starts. Every user turn is numbered. Bot-side frames carry the epoch they were made for, and every stage drops frames from a stale one.
  2. Clear goes straight to the transport. Queued audio is dropped and the client is told to flush what it buffered. Measured: 0.022 ms at p95, against a budget of 1 ms.
  3. The LLM round, the segmenter and the open TTS contexts are cancelled. Side-effect tools are the exception: they complete, and the result is reported on the next turn.
  4. The context is cut to what was played. Playback marks, playback positions and TTS word timestamps locate the last heard word. The engine waits 250 ms by default for those reports to settle.

One process. One chain of stages per call.

Calls share nothing but the inference pool, so a provider failure in one cannot touch another. Bring the STT, LLM and TTS providers you already use, over a phone line, WebRTC or a websocket.

  1. Transport in
  2. AudioIn
  3. VadWorker
  4. SttStage
  5. TurnDriver
  6. LlmRunner
  7. SegmenterStage
  8. TtsStage
  9. TransportOut
Speech to text
Deepgram, AssemblyAI, Gladia, sherpa-onnx (CPU), any server speaking the OpenAI Realtime transcription protocol
Language model
OpenAI, Anthropic, Gemini, Groq, Together
Text to speech
Cartesia, ElevenLabs, Deepgram Aura, Kokoro-FastAPI, Chatterbox
Speech to speech
OpenAI Realtime, Gemini Live

Try it without any API keys

A voice agent that runs entirely on your computer: speech recognition, language model and voice, on the CPU. It needs Docker, 4 or more cores and 8 GB of memory.

No API keys
git clone https://github.com/bilal07karadeniz/brisvox && cd brisvox
docker compose -f deploy/local/compose.yaml up
# then open http://localhost:8088/demo?force=ws

Written so an AI coding agent can pick it up cold

Claude Code, Codex, Cursor and similar tools can build on Brisvox with no prior knowledge of it.

llms.txt and llms-full.txt
The map and the whole user guide in one file, generated from the docs and checked for staleness by the quality gate.
An Agent Skill
The workflow and the common traps in two pages. Copy the folder into your project's skills directory.
brisvox-server check
Validates an agent file offline with the real loader, so an agent finds its own mistakes before anything runs.
A JSON Schema
The schema of the agent file, for editors and tools. The server prints it with --print-schema.

Common questions

Short answers with their conditions. The full list is on the FAQ page.

What is Brisvox?

Brisvox is an open-source real-time voice agent engine written in Rust. One brisvox-server process runs many isolated phone or browser calls: the caller talks, speech-to-text listens, an LLM with tools answers, text-to-speech speaks, and the caller can interrupt at any moment.

The agent is one TOML or YAML file. Your own tools run in your application through the Python or TypeScript SDK, or in Rust inside the server binary.

Introduction in the book

What license is Brisvox under, and what does it cost?

Brisvox is dual-licensed under MIT or Apache-2.0, at your option. The software is free and there is no hosted cloud, so you pay only the STT, LLM, TTS and telephony providers you choose.

Each call reports what providers bill by (LLM input, cached and output tokens, STT audio seconds, TTS characters) as a session.usage event and as Prometheus totals per provider and model. The speech test fixtures in the repository are under CC BY-SA 4.0 rather than the code license.

Does Brisvox have a hosted cloud?

No. Brisvox is self-hosted only: a prebuilt binary, a container image or a build from source, on your own machines.

How many concurrent calls can one server handle?

In the cloud capacity runs of 16 and 17 October 2026 (Linux containers on AMD EPYC 9655P), one 4 vCPU / 4 GB container held 250 concurrent calls within a p95 voice-to-voice limit of 1110 ms, and 430 with no failed call or lost turn at any latency. An 8 vCPU / 8 GB container held 470 within the same limit and 780 at any latency (p95 about 3.7 s at 780).

These runs used fake providers that speak the real Deepgram and OpenAI-compatible protocols over TLS with fixed latencies, and the real Silero VAD and Smart Turn models on every call. They measure the engine and its provider clients, not the providers. At the 4 vCPU maximum Brisvox used about 1.8 MB of memory and 0.007 cores per call.

Benchmarks

How does Brisvox handle interruptions (barge-in)?

Every user turn is a numbered epoch, and frames from a stale epoch are dropped at every stage, so audio from an answer the caller already cut off does not play. A Clear goes straight to the transport, and the LLM round, the segmenter and the open TTS contexts are cancelled.

After the barge-in the LLM context holds only the text that was actually played, so the model does not believe it said words the caller never heard.

Turns and interruptions

Which languages does Brisvox support? Does it support Turkish?

English and Turkish are supported. Turkish is a first-class language: the TTS stage rewrites written forms such as amounts, dates and times into the words a person says, and the interruption policy has Turkish backchannel and command lists. An agent can list several languages and follow the caller from one to the other in the middle of a call.

Turkish quality notes

Does Brisvox need a LiveKit or other media server?

No. The generic WebSocket, Twilio Media Streams and Telnyx media streaming transports are built in, and WebRTC and a SIP trunk listener are optional cargo features. Callers connect to brisvox-server directly.

Read the numbers, then run them yourself.

Every figure on this site comes from a result file in the repository, and each file starts with the command that produced it.

Last updated: