Machine learning
The model is a fortnight. The pipeline that keeps it fed, monitored and honest is the project.
A model that works in a notebook and never reaches production is the normal outcome, and the reason is rarely the mathematics. It is that the features computed during training cannot be computed at request time, or that they are computed slightly differently, so the model sees inputs in production it never saw in training. That gap — training and serving skew — is the single most common reason a promising model performs badly once deployed, and it is an engineering problem rather than a data science one.
Then the world moves. Prices change, a competitor launches, a season turns, and the relationships the model learned quietly stop holding. Nothing errors — the predictions simply get worse, and without monitoring on the inputs and the outputs nobody notices until someone senior asks why the recommendations look strange. Drift detection is not sophistication; it is the difference between a model you operate and a model you deployed once.
Recommendation and fraud detection deserve a particular caution: both create feedback loops. A recommender that only ever shows popular items makes them more popular and starves everything else, and the metrics look excellent while the catalogue effectively shrinks. A fraud model trained on labels produced by its own decisions learns to agree with itself. Designing for exploration and for honest labels is part of the modelling, not a refinement afterwards.
How we work
- Features are computed by one shared implementation for training and serving, which removes the most common cause of a model underperforming in production.
- Inputs and outputs are monitored for drift, because a model degrades silently rather than failing.
- A simple baseline is built and beaten before anything complicated is proposed. Often the baseline is enough.
What this includes
Pick what you need and send it over.
Questions
- How much data do we need?
- It depends far more on how many outcomes you have than on how many rows. Thousands of examples of the thing you want to predict is a workable start; millions of rows containing forty positive cases is not. That question is worth answering before any modelling begins, because it determines whether the project is possible at all.