Smart Routing Studio
AI request orchestration & cost optimisation engine
Budget-threshold routing
Route the same prompt to a different model as budget burns down
Token budget consumed
42%
3,360 / 8,000 tokens
Premium tier
Activegpt-4o
Highest quality · highest $/1M tokens
0 – 50% of budget
Value tier
nemotron-ultra-550b
Budget guardrail hit · ~62% cheaper
50 – 95%
Economy tier
qwen-2.5-72b
Hard cap · cheapest model, quality floor
95% +
Threshold events
Drag the slider or run the burn simulation — the gateway re-routes the moment a threshold is crossed.
Routing policy builder
Step 2 — choose the strategy
Provider priority
- 1nemotron-ultra-550bPrimary
- 2deepseek-r1Secondary
- 3llama-3.1-70bTertiary
- 4gpt-4oFallback
- 5claude-3.5-sonnetFallback
Visual routing flow
From prompt to response — intelligent, dynamic, cost optimised
Incoming request
OpenAI-compatible API
Intent detection
Classify & route
Complexity analysis
Token & task scoring
Cost engine
Price per 1M tokens
SLA evaluation
Tenant entitlements
Model selection
Shared vs dedicated
Execution engine
NVIDIA Dynamo
Live routing decisions
Strategy in force: Balanced
| Request | User | Intent | Selected model | Latency | Cost | Status |
|---|---|---|---|---|---|---|
| req-7843 | user_123 | Coding | deepseek-r1 | 162 ms | $0.0180 | Success |
| req-7842 | user_456 | Research | gpt-4o | 542 ms | $0.0256 | Success |
| req-7841 | user_789 | FAQ | llama-3.1-70b | 118 ms | $0.0012 | Success |
| req-7840 | user_321 | Agentic | nemotron-ultra-550b | 487 ms | $0.0312 | Success |
| req-7839 | user_654 | Vision | gpt-4o | 623 ms | $0.0487 | Success |
Traffic & cost analytics
Real-time traffic distribution
- gpt-4o32%
- Claude21%
- Gemini16%
- DeepSeek14%
- Nemotron17%
Cost optimisation
Current spend
$12,482 / day
Optimised spend
$7,104 / day