SYS-01 / Active record
Multi-tenant ML serving for Walmart Search
Modernized model serving for hundreds of millions of daily search queries while improving isolation, latency, and availability.
- Context
- Single-tenant search clusters limited resource efficiency and made consistent multi-locale operation harder.
- Decision
- Designed secure multi-tenant Kubernetes model serving with multi-tier inference routing and shared query normalization.
- Scale
- Hundreds of millions of daily queries across multi-tenant, multi-locale production environments.
- Outcome
- 99.9% availability and 25% lower inference latency, with architecture patterns adopted across Search teams.