What 100ms actually means

Working on real-time avatars at Anam has made me appreciate just how little time there is between someone finishing a sentence and expecting a reply.

We did some internal research and testing, and 200–300ms emerged as a sweet spot: a range that felt technically possible while getting close to the timing of human conversation. That was the part I was starting to get my head around. Recognising speech, deciding what to say, generating a voice and animating a face all have to fit around an incredibly short pause.

Against that range, an extra 100ms takes up between a third and half of the entire interval. A buffer or an unnecessary wait can use that much before you've improved the answer at all.

Why conversational timing is so demanding

Human conversation gives us a useful reference. In a study of turn-taking across ten languages, Stivers and colleagues found that the most common response offsets to yes/no questions fell between 0 and 200ms in each language. The average across the full dataset was about 208ms, with variation between languages and types of response.

Those timings help explain the scale of the problem. They don't establish a deadline after which a conversation stops feeling human. People pause to think, hesitate and interrupt. A thoughtful answer can take longer than a quick acknowledgement.

For an AI, getting the timing right includes knowing when to wait. Responding quickly during someone's mid-sentence pause is still getting the conversation wrong. So is continuing to speak after they've tried to interrupt.

The range we were working towards

Our internal testing pointed to 200–300ms as a useful sweet spot for conversational timing. That finding gives the engineering a reference point; it isn't a claim that every Anam response lands in that range.

Decide where the clock starts and stops

For the end-to-end conversational gap, I'd measure from the end of the user's speech to the start of the avatar's audible reply on their device.

That includes the time spent deciding the user has finished. Starting the clock only when a turn detector emits an event hides a wait the user has already experienced. At the other end, generating an audio chunk on a server doesn't mean the user has heard it yet.

It's also a different measurement from how long an avatar model takes to generate video, or how quickly a new session connects. All three matter, but a fast model or a quick connection can't tell you the full conversational response time.

With an avatar, audio and video also need to arrive in sync. A quick audible reply with a face that's visibly behind it creates another problem to solve.

Where the milliseconds go

The exact path depends on the system. A pipeline with separate transcription, language, voice and avatar models has several places where useful work can happen early, and several where it has to wait.

Audio capture and turn detection

Audio reaches the system in chunks. Capturing, buffering and transmitting those chunks takes time, some of which can pass while the user is still speaking.

The harder decision is whether they've finished. A fixed silence timeout is simple, but the system must wait out that timeout before committing to a response. Shorten it too far and a hesitation can become an accidental end of turn.

That makes turn detection both an engineering and a product problem. A lower response-time number is only an improvement if the agent still gives people room to finish.

Speech-to-text

Streaming transcription lets a system process speech as it arrives. By the end of a sentence, much of the transcription work may already be done.

Partial transcripts can change, though. Starting a response early means accepting the possibility that later words change the meaning. If the user adds “actually, don't”, the system needs to abandon the earlier answer. Speculation needs cancellation and a clear boundary before anything reaches the user or triggers an action.

Language-model inference

The useful milestone is enough text to begin a spoken reply. The first token alone may not give the voice model enough context, while waiting for the entire answer adds a delay that streaming could avoid.

Model choice, prompt length, provider load and tool calls can all affect that wait. A smaller model may help, but it needs to handle the actual task. A fast answer that misses the point is a poor trade.

Tool use can change the timing substantially. Looking up information or waiting for another service adds dependencies. A brief acknowledgement may help explain that wait, but it should be measured separately from the time to a useful answer.

Text-to-speech

A streaming voice model can start producing speech before the whole reply is written. The handoff matters: waiting for a long sentence delays the first audio, while sending tiny fragments can give the model too little context for natural delivery.

Connection setup and buffering also belong in the measurement. If the voice service needs a connection, establishing it before the first text is ready may remove a wait from the response path. The size of that saving depends on the service and setup.

Face generation

The avatar model has to turn the audio into matching facial motion quickly enough to sustain the video stream. Frame rate and response latency describe different things here. Producing frames at a steady rate doesn't tell you how long the first corresponding frame took to arrive.

Once the stream is running, different chunks can be generated, transmitted and played at the same time. That overlap keeps it moving, but the first output still has to pass through the work it depends on.

Network and playback

The user experiences the transport and playback delays as well as the models. Distance, congestion, packet loss and buffering can all add to the gap.

A jitter buffer holds some incoming media to smooth out uneven arrivals. Reducing it may shorten the delay, but can also make playback less resilient. That tradeoff needs testing on the devices and connections people actually use.

What streaming actually saves

Streaming allows the next stage to begin before the previous stage has finished the entire utterance. Transcription can happen while the user speaks. Speech generation can begin while the language model writes the rest of its reply. Video generation can begin while more audio is being produced.

That removes waits for complete inputs and moves some work before the end of the user's turn. It's a substantial part of making this kind of system practical.

First-response latency and throughput are different

In a simple streaming pipeline, the slowest stage limits the sustained rate of output. The first reply still follows a chain of dependencies. Overlapping later chunks doesn't erase the time needed to produce, transmit and play the first one.

The useful question is: what work is still on the critical path when the user stops speaking? That might be a turn-detection decision, a tool result, enough text to start speech, or a playback buffer. Streaming and faster individual stages both help when they shorten that path.

What this means for the product

My focus at Anam is the product experience across Lab and the SDK. Understanding these timings matters because the product is where people encounter the combined result of all that engineering. A developer can make sensible choices about each component and still end up with a conversation that feels slow.

I'd want to inspect the full experience in a few ways:

Trace the handoffs. Capture turn detection, transcription, first usable text, first audio and client playback timings. Use a common clock where possible; timestamps from different machines need clock synchronisation before subtracting them.
Look at the distribution. The median describes a typical turn; p95 helps expose the slower ones. Break results down by model, region, device and whether a tool call was needed.
Test who gets to speak. Include pauses, corrections and interruptions. A response-time improvement that makes the agent talk over people needs another look.
Listen on real connections. Test mobile devices, distant users and poor networks, including packet loss. Browser request throttling alone doesn't establish how a WebRTC media stream behaves under those conditions.

This is part of what excites me about building at Anam. We're working on AI avatars that people can actually have a conversation with, and getting closer to natural timing requires care across the whole product. There are gains to make in the models, in how the stages connect and in the experience developers build with them.

You can try a conversation in Anam Lab. Pay attention to the handover when you finish speaking, pause or change your mind. Those are the moments I'm thinking about when I say that 100ms matters.