“Model-agnostic” is a claim every platform makes and almost none can demonstrate, because the demonstration only exists under load. We are a top-250 Anthropic customer by token volume, we route across multiple frontier providers, and our customers get new models the day they exist with zero migration work. This is how that actually functions.
Routing is by task shape, not by model brand. The core classifies every unit of work into tiers — deep reasoning, long-context synthesis, fast mechanical transforms — and each tier maps to whatever model currently wins it on our evals. Agents and procedures address the tier, never the model. That indirection is the entire trick: when a new frontier model ships, we run it against the tier evals, flip the mapping where it wins, and every deployment inherits the upgrade. Customers have watched a new model arrive in their VPC the week it launched, with no change to a single procedure.
Fallbacks are a correctness feature, not an uptime feature. Provider incidents are routine at enterprise scale — a degraded endpoint at 2 a.m. during a customer’s Monday-reporting run is not an acceptable reason for the Monday report to not exist. Every tier has an ordered fallback chain, and the router demotes automatically on latency and error signatures. The subtle part is semantic drift between models on the same task; anything governed by an exacting output contract (a customer’s template, a working paper format) validates the artifact, not just the HTTP status, before accepting a fallback’s answer.
The cost curve is the quiet win. Frontier pricing falls unevenly — a tier that cost X in January costs a fraction of it by summer if you re-run the evals and re-map aggressively. Because routing is centralized, that saving lands on every deployment simultaneously, including air-gapped installs on their next bundle. A customer’s technical leadership once articulated our ROI back to us unprompted, in exactly these terms: the platform gets better and cheaper underneath their teams while their procedures stand still.
What we do not do: chase leaderboard deltas. A model that wins a tier by two points but loses on instruction fidelity under our governance harness does not ship. The eval suite — not the announcement cycle — is the gate.
Question 11 on every enterprise diligence list is “which models, and what about the next one?” This page is the answer we give in public.
— Platform eng, New York