AI agents
A demo agent needs to be impressive once. A production agent needs to be wrong in ways you can detect, bound and recover from, several thousand times a day.
The distance between a convincing demo and something you can put in front of customers is almost entirely about failure. A model will confidently do the wrong thing, and the question is not whether but what happens next. That means every action an agent can take is a named tool with a typed contract, not free text executed hopefully; that irreversible actions are gated behind confirmation or a human; and that the agent's reasoning and tool calls are recorded, because an agent you cannot audit is one you cannot debug or defend.
Most useful agents are less autonomous than the word suggests, and that is a feature. The shapes that work are narrow: triage an inbound queue and draft a reply for a person to send; take a defined workflow with clear inputs and outputs and run it end to end; answer from a corpus you control, with citations. The shapes that disappoint are the open-ended ones — an agent given broad authority and a vague objective will produce plausible work that nobody can check, which costs more to review than it saved.
And an agent without evaluation is a rumour. Before it ships there is a set of real cases with known-good outcomes, so a prompt change or a model upgrade produces a number rather than an argument. This is the piece most often skipped, and it is the piece that decides whether you can improve the thing after launch or merely keep changing it.
How we work
- Every capability is a typed tool with explicit permissions. An agent cannot do anything you have not deliberately handed it.
- Irreversible actions — money, messages to customers, deletions — need a confirmation step or a person, every time.
- An evaluation set of real cases exists before launch, so changes are measured rather than argued about.
- Every run is traced: the inputs, the tools called, the output. Anything else is unauditable.
What this includes
Pick what you need and send it over.
Questions
- Will it make things up?
- Sometimes, yes — that is a property of the technology, not a bug to be promised away. The engineering answer is to constrain what it can assert: answer from retrieved sources with citations, keep the model away from arithmetic and lookups that code should do, and design the interface so a person can see what it relied on.
- Which model should we use?
- Whichever one passes your evaluation set most cheaply, and the answer changes every few months. That is precisely why the system is built so the model is a swappable component rather than something the whole product is welded to.