Every model is sold as the best one. They are not interchangeable. One reasons carefully, one answers fast, one handles the messy, open-ended questions. You cannot feel that difference from a benchmark table, and picking wrong is expensive twice: once in the bill, once in the answers.
So I built a chat platform with three models behind one window: Gemini, DeepSeek-R1 and Llama-3.1-8B. Same prompt, same screen, three honest answers. Here is what it takes to make that work without turning three distinct models into one bland one.
Three models, three personalities
- Gemini for complex, open-ended queries where breadth matters.
- DeepSeek-R1 for problems that reward careful step-by-step reasoning. It thinks before it answers, which makes it slower and often more right.
- Llama-3.1-8B for fast, direct responses. Small, quick and cheap, and for a surprising share of everyday questions it is all you need.
Seeing them answer the same awkward question side by side teaches more about model choice than any comparison article, this one included.
One conversation format, many APIs
Every provider has its own idea of what a conversation looks like. Role names differ (some call the model's turns assistant, Gemini calls them model), system prompts live in different places, and streaming responses arrive in different shapes.
The fix is an old one: define your own internal conversation format and write a thin adapter per provider that translates in and out of it. The rest of the application never learns which model it is talking to.
class ChatAdapter:
"""Every provider is wrapped to look exactly like this."""
def stream(self, messages, system=None):
"""messages: [{"role": "user" | "assistant", "content": str}]
yields: {"type": "text" | "reasoning" | "done", "content": str}"""
raise NotImplementedError
class GeminiAdapter(ChatAdapter):
def stream(self, messages, system=None):
contents = [
{"role": "model" if m["role"] == "assistant" else "user",
"parts": [{"text": m["content"]}]}
for m in messages
]
# ...call the API with contents and system,
# translate each streamed chunk into the shared event shape...
ADAPTERS = {"gemini": GeminiAdapter(), "deepseek": ..., "llama": ...}Normalise the plumbing, never the personality. The adapter should change how a model is reached, not what it says.
Handling a model that thinks out loud
Reasoning models like DeepSeek-R1 produce a chain of thought before the final answer. Depending on where the model is hosted, that reasoning arrives in a separate field or inline inside <think> tags. Pass it straight through and users see a wall of internal monologue. Strip it silently and you throw away the most interesting thing about the model.
I treat reasoning as its own event type. The adapter separates it from the answer, and the interface shows it collapsed by default under the reply, so anyone curious can open it. Also worth knowing: the reasoning should not be fed back into the conversation history on the next turn. It costs tokens and the provider generally does not expect it.
Streaming, timeouts and graceful failure
Three models means three chances for something to go wrong on every message: a rate limit, a timeout, an overloaded endpoint. A few rules keep that from breaking the experience:
- Stream everything. Waiting twenty seconds for a reasoning model to finish before showing anything feels broken. Tokens arriving immediately feel fast, even when the total time is the same.
- Fail per model, not per page. If one provider errors, show that one panel's error and let the other two carry on.
- Translate errors into plain language. "This model is busy, try again in a minute" is useful. A raw JSON error body is not.
- Set a timeout per model. A reasoning model legitimately takes longer than a small fast one, so a single global timeout is either too short for one or too long for the other.
Keeping the bill sane
Long conversations get expensive quietly, because every turn resends the whole history. Trim old turns once a conversation passes a sensible length, cap the maximum response size per model, and rate limit per visitor. API keys stay on the server and never touch the browser, which sounds obvious and is still the most common mistake I see in public demos.
What it taught me about choosing models
- Start with the smallest model that might work. Move up only when you can point to the questions it gets wrong.
- Build the adapter layer even if you only use one model today. It makes switching a configuration change instead of a rewrite, and you will switch.
- Judge models on your own questions. Keep a short list of real prompts from your domain and run every new model against it. That list will outlive every leaderboard.
One contract, three models
The interesting engineering in a multi-model platform is not calling three APIs. It is building one clean contract that all three fit behind, while letting each model stay what it is. The platform is live on my projects page. Ask all three the same awkward question and the choice makes itself.
