LLM integration
Adding a model to an existing product is an afternoon. Making it behave the same way tomorrow is the actual work.
The first version always works. Someone wires an API call into a feature, the output is impressive, and the demo goes well. Then the same input produces a different answer, a longer document overflows the context, a customer pastes something adversarial, and the provider ships a new model version that quietly changes the tone of everything. None of these are exotic. They are Tuesday, and a system that has not planned for them is a prototype regardless of who is using it.
Most of the quality lives in context rather than in prompt wording. What the model is shown — which documents, in what order, trimmed how, with what already established — determines the answer far more than clever phrasing does. That makes context assembly a piece of software with its own logic and its own tests, not a string that grew organically until it worked. The prompt itself should be versioned like code, because it is code.
And you cannot improve what you do not measure. A regression suite of real inputs with known-acceptable outputs turns "the new model seems worse" into a number, which is the difference between tuning a system and redecorating it. It is also the only way to change providers without a leap of faith — and you will change providers, because the price and capability landscape moves every few months.
How we work
- Context assembly is real software with tests, not a string that grew until it worked.
- Prompts are versioned and reviewed like any other code, because a change to one is a change to behaviour.
- A regression suite exists before launch, so swapping providers is a measurement rather than a leap.
What this includes
Pick what you need and send it over.
Questions
- Should we self-host a model?
- Rarely, and almost never first. It makes sense when data genuinely cannot leave your network, or at a volume where the arithmetic clearly favours it. Otherwise you are taking on GPU capacity planning and model operations in exchange for a bill that is often larger, not smaller.