Accuracy claims for document extraction are not very useful. Ninety-something percent of what, measured how, on which documents? What a credit analyst wants to know is where it breaks, because that tells them what to check. So here is where it breaks, in our experience, on credit agreements and indentures. These are the failure modes we test against, and the ones we would ask any vendor, or any internal team, to show their results on.
1. The threshold is not the number on the page
A leverage covenant reads "shall not exceed 5.00 to 1.00". Straightforward, until you notice the step-down schedule two pages later that takes it to 4.50 from the sixth test date, the springing condition that only tests it when the revolver is more than 35% drawn, and the equity cure that lets the borrower add cash to EBITDA twice in any four quarters. An extraction that returns 5.00x has found the headline and missed the covenant. The field is not a number; it is a schedule with conditions, and the test date decides which row applies.
2. Baskets stack, and the stacking is the capacity
The general debt basket permits the greater of 100 and 25% of EBITDA. The ratio basket permits unlimited debt below 4.0x. The incremental facility has its own free-and-clear amount and its own ratio prong. These are not three separate facts. Debt can be incurred under any combination, reclassified between them, and the "available amount" builder for restricted payments may grow with retained excess cash flow and shrink with each use. An extraction that lists the baskets has done the easy part. The hard part is the model of how they combine, and that is where most first passes stop.
3. Defined-term chains
EBITDA is defined by reference to Consolidated Net Income, which excludes items defined by reference to the Transactions, which are defined in the recitals. Permitted Liens references Permitted Refinancing Indebtedness, which references the debt it refinances. A definition can be five links deep and can cross into a different document via the intercreditor. Extraction that captures each definition as a standalone paragraph has the words but not the meaning. The value of a term depends on the chain, and the chain has to be followed and stored, with each link cited.
4. Grower baskets
"The greater of USD 50 million and 20% of Consolidated EBITDA" is a basket whose size changes every quarter. At close, the fixed amount governs. Two years of growth later, the EBITDA leg does. An extraction that stores 50 million as the cap is right on day one and wrong thereafter, and it is wrong in the direction that understates borrower flexibility, which is the direction that surprises lenders. The field has to hold both legs and be re-valued on every certificate.
5. Amendments and what they replaced
The fourth amendment restates section 7.01 in full. The first pass extracts the original and the restated version and, without help, cannot say which is in force. Worse, an amendment can change a definition without touching the covenant that uses it, so the covenant text is unchanged and its meaning is not. Extraction has to be document-aware: which instrument, effective from when, superseding what.
6. Cross-references that leave the document
"As defined in the Intercreditor Agreement". "Subject to the terms of the Security Agreement". "The amount set out in the Fee Letter". Some of the numbers that decide capacity are not in the agreement being read. The correct extraction result is an explicit gap with a pointer, not a guess. Systems that always return a value are the dangerous ones.
What good looks like
Every one of these has the same remedy: the extracted field is a structure, not a scalar; every element of the structure carries a verbatim quote and a page; and anything the machine cannot verify against the page is routed to a person rather than written into the register. Ask to see the register for a document with a step-down, a grower and a restated covenant. If the vendor has done the work, it will show. If they quote an accuracy percentage instead, you have your answer.