If you run a credit function you have been pitched AI several times this year, and you were right to be sceptical each time. Not because the technology is weak, but because most pitches do not say where it sits in the process, and in a control function that is the only question that matters. Here is where we have put it, and where we have kept it out.
The dividing line
The rule we use is simple. Anything whose answer a reviewer would expect to reproduce exactly, from the same inputs, is deterministic code. Anything whose value lies in judgement, ranking or language is a candidate for a model, and the model's output is advisory until a person or a deterministic check accepts it.
| Deterministic, always | Model-assisted, with a check |
|---|---|
| Headroom: the covenant formula applied to the reported figures, on the basis the agreement specifies | Reading the agreement and proposing the formula, the basis and the threshold, each with a verbatim quote |
| Due dates: derived from the reporting frequency and the delivery lag in the agreement | Recognising which document has arrived and which fund and period it belongs to |
| Exposure: positions, limits and utilisation, from the systems of record | Flagging that a NAV notice is indicative rather than official, or that dealing has been suspended |
| State transitions: due, overdue, stale, cured, breached | Prioritising the morning list when there are forty items and time for ten |
| Limit proposals: the policy rule applied to the validated NAV | Drafting the chase email and the case narrative for the escalation |
| The audit trail | Summarising a quarter of activity on a fund for the annual review |
Nothing in the left column ever calls a model. If the leverage ratio is 4.8x against a 5.0x threshold, that number came from arithmetic over cited inputs and will be the same number tomorrow. If a fund is marked stale, it is because a date passed, not because something inferred it.
Where the model does its work
Extraction is the obvious place, and it is where the value is: reading a schedule and turning "declines by more than 15% over any three-month period, excluding subscriptions and redemptions" into a test with a type, a window, a threshold and a basis. But the extracted field is a proposal. It is checked against the page by code, it is flagged if the check fails, and it is confirmed by a person at onboarding. From then on it is data.
The other places are the ones where a person would otherwise be reading and ranking. Which of today's forty items matters most, given exposure, headroom and how long each has been open. What the story is on a fund that has had three late deliveries and one near-breach this year. What the chase email should say, given the last two went unanswered. These are drafts and orderings. They save time, they are visibly labelled, and nothing downstream depends on them being right.
The model may read, propose, rank and draft. It may not compute a figure the bank relies on, move a date, or change a state.
Why the line is there
Three reasons, in the order the buyer usually raises them.
- Auditability. A second-line function has to show its working. Arithmetic over cited inputs can be shown. A model's reasoning cannot, in the sense an auditor means, so it must not be in the chain that produces a number.
- Reproducibility. Run the same certificate through the system twice and you get the same headroom. That is a property of code, not of models, and it is the property that lets you re-run history when a threshold is corrected.
- Failure shape. When code is wrong it is wrong the same way every time, which is findable. When a model is wrong it is wrong occasionally and plausibly, which is not. Occasional, plausible errors belong where a person will see them, not in a limit.
What to ask a vendor
Ask for the list. Which outputs are computed and which are generated, and for each generated one, what checks it before it is used. If the answer is a diagram of an agent, keep asking. If the answer is a table like the one above, you can decide whether you agree with where the line is. We think it is in the right place; we would rather argue about that than about accuracy percentages.