Mercury 3: Measure the Wait, Not Just Tokens per Second

What changed with Mercury 3

Inception positions Mercury 3 for latency-sensitive work, including support and web agents, and says it is available through its API and distribution partners. Its documentation describes diffusion as generating tokens in parallel and provides an OpenAI-compatible interface. Compatibility can make an integration trial easier; it does not establish equivalent behaviour in your workflow. [1][2]

Treat launch benchmarks and customer testimonials as reasons to investigate, not promises that transfer unchanged to your own traffic. The deployment route, request size, tool dependencies and acceptance standard should all be recorded alongside a result.

Put separate clocks on separate kinds of waiting

Baseten's inference guide distinguishes total request latency, time to first token, inter-token latency and perceived output speed. These measure different parts of the experience. A system can start quickly but finish slowly, or generate rapidly after a long initial wait. [3]

For a business trial, add two application-level measures. Time to first useful answer starts with the user's request and ends when the interface communicates something that advances the task. Time to accepted completion ends only when the agreed outcome is delivered and verified. These are proposed operating measures, not published Mercury 3 benchmarks.

An acknowledgement such as 'checking that now' may reassure the user, but it should not count as the useful answer. Nor should a completed model response count as a completed booking if the booking tool has not confirmed it.

Trace a customer request through the whole workflow

Consider a service customer asking for the earliest available appointment. The system may need to retrieve the customer's details, check a calendar, prepare a response and confirm a reservation. Faster generation affects only some of that work. Use fictional customer records during an initial test, with no live bookings or external messages.

Record the start and finish of each stage, then compare the same request across candidate configurations:

• Model time: capture the initial wait and the time spent generating the answer.
• Tool time: measure customer lookup, calendar access, retries and confirmation separately.
• User time: include clarification, approval and any correction needed before the task is accepted.
• Failure outcome: record whether a timeout produces a clear explanation, a safe retry or an uncertain result requiring staff review.

Choose the improvement your customers will notice

If calendar access dominates the delay, a model switch alone may have little effect. If long generated responses dominate, a faster model or a shorter answer may help. The trace identifies which hypothesis to test instead of attributing every delay to the model.

Repeat representative requests under normal and busier conditions. Report unusually slow cases as well as typical ones, and retain unsuccessful attempts rather than quietly excluding them. Compare cost per accepted task with the quality and permissions checks unchanged.

Before expanding a trial, agree what counts as an acceptable answer and who handles exceptions. Ask suppliers which speed measurements include networking, queueing and tool calls. The purchasing question is not simply which model produces the most text each second. It is which configuration delivers the required result promptly and reliably for the people using it.

Sources

• Inception: Introducing Mercury 3 (19 October 2026) - First-party launch, availability and vendor-reported generation performance.
• Inception platform documentation (Accessed 20 October 2026) - Diffusion model approach, API compatibility and documented platform capabilities.
• Baseten: The Inference Stack (27 January 2026) - Infrastructure provider's explanation of distinct latency metrics and production performance.

Continue learning

• Understand the whole business AI workflow
• AI workflows for service businesses
• More ELYMENT AI Insights

Frequently asked questions