What we learned putting 70+ models behind one interface

Quad Chat started from a simple idea: people should not need a separate subscription, a separate interface and a separate set of habits for every AI model they want to use. One subscription, one place, more than seventy models. The idea is simple. Making it work well has been one of the most instructive engineering problems we have taken on, because it forced us to confront, in detail, how different these systems really are beneath their similar-looking chat boxes. This post collects some of what we learned.

Normalizing APIs that only look alike

At a glance, most model APIs share a shape: send a list of messages, receive a response. The resemblance is real enough that it is tempting to write a thin adapter per provider and call it done. We learned quickly that the resemblance ends at the surface.

Providers differ in how they represent roles, how they handle system instructions, whether they accept multiple system messages or only one, how they encode images and files, what they do with empty or consecutive messages from the same role, and how they report errors. Each of these differences is small. Together, they mean that a conversation which is perfectly valid for one model can be rejected or subtly misinterpreted by another.

What worked for us was an internal representation of a conversation that is richer than any single provider's format, with explicit translation into each provider's expectations. Translation is where the knowledge lives. It is also where we write the most tests, because a bug there does not crash anything; it just quietly changes what the model sees. The principle we settled on is that the internal format should describe what the user and the product intend, and each adapter should be responsible for expressing that intent faithfully, even when a provider has no direct equivalent.

Streaming and tool calling

Streaming is essential to how a chat product feels. People want to see an answer begin immediately rather than wait for it to finish. But providers stream in different formats, with different event types, different ways of signaling the end of a response, and different behavior when something goes wrong partway through. A stream that stops unexpectedly might mean the answer is complete, the connection dropped, or the provider hit a limit. The product needs to tell these apart, because each calls for a different response.

We ended up treating streams as sequences of normalized events rather than raw text, and building the interface around those events. That made it possible to show partial answers consistently, to recover more gracefully from interruptions, and to display intermediate states such as a model searching the web or calling a tool.

Tool calling is where differences run deepest. Models vary in how they declare tools, how they format arguments, whether they can call several tools at once, how reliably they follow a schema, and how they expect results to be returned. Some models are careful and precise with tools; others are more improvisational. A feature built on tools, like web search or the app builder, has to be designed so that it degrades sensibly when a model uses tools imperfectly, rather than assuming every model behaves like the best-behaved one.

A bug in translation does not crash anything; it just quietly changes what the model sees.

Context windows, tokens and switching mid-conversation

One of the features people value most is the ability to switch models partway through a conversation: start with one model, then ask another for a second opinion or a different style. It is also one of the hardest features to do well.

Models have different context window sizes, and they count tokens differently, because they use different tokenizers. The same conversation can be comfortably within one model's limit and well over another's. Text in some languages, including Albanian, can be tokenized much less efficiently by some tokenizers than English, which means the effective context available for those users can be smaller than the headline number suggests. We had to stop thinking in terms of a single token budget and start estimating per model.

When a conversation does not fit, something has to give. We have to decide what to keep, what to summarize, and what to drop, and we have to do it in a way that preserves what the user most likely cares about. There is no perfect answer. What we have found is that it helps to be transparent: if the model cannot see the whole conversation, the product should not pretend that it can.

Switching also raises questions beyond context. Models have different capabilities, so a conversation containing images or tool results may not translate cleanly to a model that cannot handle them. And models have different personalities, so a switch can feel jarring even when it works technically. We try to make the switch explicit in the interface, so people understand that a different system is now answering, and to carry forward as much of the conversation as the new model can meaningfully use.

Outages, fallbacks, cost and latency

Depending on many providers means inheriting all of their availability. Any external service can slow down, return errors or become temporarily unavailable, and with enough providers, some part of the system is usually having a harder day than the rest. The product has to be built on the assumption that failure is normal.

We treat fallbacks as a product decision, not only an infrastructure one. Silently routing a request to a different model when the chosen one is unavailable can keep things working, but it also changes the answer the person receives without their knowledge. When someone has deliberately chosen a model, we would rather tell them it is unavailable and offer alternatives than substitute behind their back. When they have not expressed a preference, we have more latitude. Being explicit about the difference has made these decisions much easier.

Cost is another constant. Models vary widely in what they cost to run, and a single subscription covering many of them has to account for that honestly. We try to design so that people can use expensive models when they need them, while making sure the system itself does not waste resources: avoiding unnecessary retries, not resending more context than a request needs, and choosing lighter models for internal tasks like titling a conversation.

Latency interacts with all of this. The time before the first word appears shapes how fast a product feels more than total generation time does. Different models and providers have very different latency profiles, and those profiles change. We measure them continuously rather than relying on assumptions, because what was true last month may not be true now.

Which model is right for which task

People often ask us which model is best. The honest answer is that it depends on the task, the language, the person's preferences, and the moment. Models that excel at careful reasoning may be slower than people want for a quick question. Models that write fluently may be less reliable with facts. A model that is strong in English may be noticeably weaker in Albanian.

We have found public benchmarks useful as a rough guide and insufficient as a basis for decisions. What matters more is how models perform on the kinds of requests people actually bring to the product, in the configuration the product actually uses. We are building evaluation methods oriented around that: tasks drawn from real categories of use, tested in multiple languages, judged with attention to accuracy and honesty as well as fluency. It is ongoing work, and it is one of the threads that led directly into our research program.

What this taught us about building our own models

Working with so many models has shaped how we think about building our own. We have seen that capability is only part of what makes a model useful. Consistency matters: a model that behaves predictably with tools and instructions is easier to build on than a more capable one that surprises you. Efficiency matters: fast, inexpensive models make features possible that would otherwise be impractical. Language coverage matters: the gap between how models serve English speakers and how they serve Albanian speakers is visible to us every day.

We have also learned humility. The providers behind these models are solving extraordinarily hard problems, and seeing the variety of approaches up close has taught us a great deal. Our aim is not to compete across the board, but to contribute where we can see a real gap: models and methods for speech, for less-served languages, for efficient deployment, and for evaluation that reflects how people actually use these systems.

Putting seventy models behind one interface turned out to be a way of learning what a good model, and a good product, really needs. We are still learning, and we expect to write more as we do.