The model shelf: which models we run, and why

The fastest way to overspend on AI is to send every task to the biggest model. We keep a shelf of models and reach for the smallest one that does the job — local by default, hosted only when there’s a clear reason.

The shelf

Job	What we run	Why this one
Routing / intent / tools	Small instruct model (2–3B)	Millisecond classification; cheap enough to run on every turn
Generation (grounded)	Mid-size open model, low-temp	Quality answers, constrained to retrieved sources
Embeddings	Multilingual embedding model	One vector space across English and Indian languages
Reranking	Cross-encoder reranker	Puts the most relevant context at the top, not just the most similar
Speech (ASR / TTS)	Indic speech models	Rural, multilingual UX where typing isn’t the interface
Translation	Indic translation model	Serve content in the language the user thinks in
Image generation	Flux / SDXL on our own GPU	Original assets, no per-image SaaS, no egress

The gateway

Everything sits behind a single LiteLLM gateway. Local models (served via Ollama or vLLM) are the default route; the gateway can fall back to a hosted provider for non-sensitive, opt-in work, with cost tracked per call. Swapping a model is a config change, not a code change — the application only ever talks to the gateway.

The rule

Smallest model that works. Local for anything sensitive. Hosted only when the quality gap is real and the data is allowed to leave.

That discipline is why a routing decision doesn’t cost what a generation costs, and why a client’s confidential corpus never touches someone else’s API.

Dogfooding

Every illustration on this site — the case-study covers, the AI-first imagery, the backdrop on the home page — was generated on our own RTX 4090 with Flux and SDXL, through the same sovereign engine we build for clients. The patterns we sell are the patterns we run.