LATENCY01 / 04
Find latency before
users feel it.
Separate queue time, provider delay, time-to-first-token, generation speed and total duration.
Queue42ms
Policy18ms
Provider158ms
Generation394ms
Understand the behavior.
Latency, economics, evaluation and reliability answer different questions. Explore how those signals can be examined together without mistaking one metric for the whole system.
02 / INFERENCE INTELLIGENCEKnow why your inference
Know why your inference
behaves the way it does.
A technical view of the signals an inference control plane can bring together. All figures and states below are illustrative scenarios.
Find latency before
ECONOMICS02 / 04
Understand cost
down to the request.
Attribute usage and examine the shape of model, output, retry and routing costs.
EXAMPLE MONTHLY INFERENCE$12,482.16
↘ 18.4%scenario deltaModel 62%Output 21%Retries 9%Routing 8%
QUALITY03 / 04
Compare models with
contextual quality signals.
Relate evaluation dimensions to the task. A fast response is not automatically a useful one.
RELIABILITY04 / 04
Make provider
degradation visible.
Understand health as a changing signal, not a permanent property of an endpoint.
OpenAIHealthy
AnthropicHealthy
GeminiElevated latency
BedrockHealthy
vLLM ClusterHealthy
INTERPRETATION
Separate signal from conclusion.
- Latency is not quality
- A fast response may fail the task. Latency and task-specific evaluation belong in separate dimensions before being considered together.
- Cost needs context
- Input length, output length, model choice, retries and cache conditions can change costs. The economics panel uses example values, not Orbitriva customer savings.
- Reliability changes over time
- A healthy sample is not proof of sustained availability. Provider states on this site are illustrative; no health endpoints are connected.