The shim groups every call into workloads by service, prompt template and tool use, then sorts each one by how predictable it is. That decides which lane it can take.
| Workload | Class | Code calls | Route | Calls/day | Unrouted | Actual | Saved | Parity | Cost · 30 days |
|---|---|---|---|---|---|---|---|---|---|
Fleet ops copilot ops-copilot · Engineering | variable | Claude Opus 5 Anthropic SDK | Decision model Haiku 4.5 / Sonnet 5 / Opus 5 · since Mar 13 | 25.7K | $45.5K | $16.3K | $29.2K −64% | 99.1% | |
Fleet telemetry summaries fleet-ops · Data Platform | consistent | GPT-5.6 Sol OpenAI SDK | Market route GPT-5.6 Luna · since Sep 14 | 56.2K | $31.6K | $3.37K | $28.2K −89% | 97.9% | |
Support ticket triage support-bot · Customer Support | consistent | Claude Sonnet 5 Anthropic SDK | Market route Qwen3.8 27B · since Aug 5 | 136.2K | $28.4K | $1.10K | $27.3K −96% | 99.7% | |
PR code review bot ci-reviewer · Engineering | variable | Claude Opus 5 Anthropic SDK | Decision model Kimi K2.7 Code / Sonnet 5 / Opus 5 · since Nov 23 | 11.7K | $31.2K | $12.7K | $18.5K −59% | 99.1% | |
Perception frame captions perception-captioner · Research | consistent | Gemini 3.1 Pro Gemini SDK | Market route Gemini 3.8 Flash · since Dec 3 | 153.8K | $21.4K | $7.73K | $13.7K −64% | 97.8% | |
Training-data labeling label-pipe · Research | specialty | Claude Sonnet 5 Anthropic SDK | A/B test control vs. challenger · since Sep 5 | 68.9K | $16.1K | $14.6K | $1.48K −9% | testing | |
Sim task planner sim-planner · Research | specialty | Gemini 3.1 Pro Gemini SDK | A/B test control vs. challenger · since Sep 2 | 8.7K | $10.8K | $10.7K | $62 −1% | testing | |
Incident root-cause agent oncall-agent · Engineering | variable | Claude Fable 5.1 Anthropic SDK | Observing shimmed Jun 28 | 1.7K | $21.2K | $21.2K | — | — | |
Internal knowledge answers kb-answer · Product | variable | GPT-5.6 Sol OpenAI SDK | Observing shimmed Jul 29 | 34.3K | $10.8K | $10.8K | — | — | |
SQL analytics copilot data-copilot · Data Platform | variable | Claude Sonnet 5 Anthropic SDK | Observing shimmed Jul 25 | 15.7K | $7.90K | $7.90K | — | — | |
Contract & invoice extraction finops-extract · Finance & Ops | consistent | Gemini 3.1 Pro Gemini SDK | Pinned shimmed Feb 13 | 6.5K | $5.72K | $5.72K | — | — | |
Customer email drafts crm-assist · Sales & Marketing | consistent | GPT-5.6 Sol OpenAI SDK | Observing shimmed Jul 21 | 24.9K | $5.53K | $5.53K | — | — |
| Seller | $/Mtok in·out | p50 | Uptime | Traffic now |
|---|---|---|---|---|
On-prem · DAL-2 Cheapest · primary · 71% busy | $0.09 · $0.28 | 540ms | 99.91% | 68% |
Vast.ai · spot Spillover · deep | $0.17 · $0.60 | 690ms | 99.62% | 27% |
OpenRouter Spillover · deep | $0.22 · $0.74 | 720ms | 99.78% | 5% |
AWS Bedrock Fallback only | $0.30 · $0.90 | 610ms | 99.95% | 0% |
| Time | Workload / prompt | Lane | Difficulty | Sent to | Tokens | Latency | Cost | Grade |
|---|---|---|---|---|---|---|---|---|
| 16:42:18 | Fleet ops copilot Unit PX-118 stuck in recovery loop at DAL-2; check last 3 deploys and propose a fix… | Decision model | 0.60 | Claude Sonnet 5 | 14,471 | 4,435ms | $0.0382 was $0.0955 | grading… |
| 16:42:14 | Support ticket triage Classify ticket #48213: "robot arm stalls after firmware 4.2…" → {category, urgency, team} | Market route | — | Qwen3.8 27B via Vast.ai (open-weight) | 3,413 | 827ms | $0.0004 was $0.0105 | grading… |
| 16:42:12 | Internal knowledge answers What's the torque limit on the Gen-3 wrist actuator and where's the spec? | Pass-through | — | GPT-5.6 Sol | 6,962 | 1,429ms | $0.0164 | grading… |
| 16:42:08 | Perception frame captions Describe frame 18842 from cam-3 (bin picking cell 2): objects, occlusions, anomalies… | Market route | — | Gemini 3.8 Flash | 1,809 | 314ms | $0.0019 was $0.0056 | 100 |
| 16:42:04 | Internal knowledge answers What's the torque limit on the Gen-3 wrist actuator and where's the spec? | Pass-through | — | GPT-5.6 Sol | 3,135 | 2,909ms | $0.0116 | 98 |
| 16:42:00 | Support ticket triage Classify ticket #48213: "robot arm stalls after firmware 4.2…" → {category, urgency, team} | Market route | — | Qwen3.8 27B | 3,635 | 782ms | $0.0004 was $0.0085 | 93 |
| 16:41:56 | Perception frame captions Describe frame 18842 from cam-3 (bin picking cell 2): objects, occlusions, anomalies… | Market route | — | Gemini 3.8 Flash | 1,887 | 249ms | $0.0018 was $0.0050 | 100 |
| 16:41:53 | Training-data labeling Label grasp outcome for clip #900213 (frames 40–88): success / slip / drop… | A/B test | — | Claude Sonnet 5 | 4,133 | 1,421ms | $0.0094 | 90 |
| 16:41:49 | Support ticket triage Classify ticket #48213: "robot arm stalls after firmware 4.2…" → {category, urgency, team} | Market route | — | Qwen3.8 27B via Vast.ai (open-weight) | 3,425 | 668ms | $0.0003 was $0.0080 | 96 |
| 16:41:47 | Training-data labeling Label grasp outcome for clip #900213 (frames 40–88): success / slip / drop… | A/B test | — | Claude Sonnet 5 | 3,245 | 1,583ms | $0.0096 | 94 |
| 16:41:44 | Perception frame captions Describe frame 18842 from cam-3 (bin picking cell 2): objects, occlusions, anomalies… | Market route | — | Gemini 3.8 Flash | 2,076 | 315ms | $0.0022 was $0.0062 | 93 |
| 16:41:41 | Contract & invoice extraction Extract {vendor, PO, line items, terms} from invoice_2026_09_118.pdf… | Pass-through | — | Gemini 3.1 Pro | 9,609 | 2,146ms | $0.0233 | 93 |
| 16:41:37 | Perception frame captions Describe frame 18842 from cam-3 (bin picking cell 2): objects, occlusions, anomalies… | Market route | — | Gemini 3.8 Flash | 1,630 | 329ms | $0.0016 was $0.0046 | 100 |
| 16:41:34 | Customer email drafts Draft follow-up to {account} re: pilot renewal, tone=warm, include ROI recap… | Pass-through | — | GPT-5.6 Sol | 3,148 | 2,339ms | $0.0111 | 100 |
| 16:41:31 | Support ticket triage Classify ticket #48213: "robot arm stalls after firmware 4.2…" → {category, urgency, team} | Market route | — | Qwen3.8 27B via Vast.ai (open-weight) | 3,496 | 711ms | $0.0004 was $0.0086 | 100 |
| 16:41:29 | Support ticket triage Classify ticket #48213: "robot arm stalls after firmware 4.2…" → {category, urgency, team} | Market route | — | Qwen3.8 27B via Vast.ai (open-weight) | 2,855 | 1,141ms | $0.0003 was $0.0094 | 95 |