Every few weeks a product claims to read filings faster than a person. The claim is nearly true, which makes it more demanding to check than a false one would be. On earnings text, language models fail in recognizable places with a recognizable geometry; this piece tours the four largest and ends with the questions any vendor should survive.
Boilerplate, the carrier wave
Filings are overwhelmingly template by weight. The same paragraphs open countless outlooks — we remain confident in our strategy — and a model raised on that ocean inherits its indifference. When real news enters a passage, the uninformative surroundings do not move aside for it, and a similarity score measured against the whole corpus will call a changed sentence familiar company.
The partial defence is comparing a passage only against its previous self and near neighbours, a narrow diff that ignores the generic bulk of the language. Even so, drift hidden by boilerplate remains the most common silent failure: scores stay warm while the meaning walked out.
The euphemism gradient
Retrenchment arrives in verbs chosen raw by nobody: streamline, right-size, optimize, rebalance. A model tutored on so much soft language will grade it soft — scoring the laid-off as mildly as the prose graded the reader. Tone systems mistake register for content, and the mistake is systematic in one direction, which is worse than noise: it lends inertia exactly where vigilance pays.
Comparative scoring with a fixed glossary of softening terms is a crude correction, and it works well enough to be a professional habit. The deeper courtesy is naming: when the tables say the departure costs are permanent, a reading that still says streamline has a euphemism gradient, not a finding.
Recast one-offs, the quiet migrations
Accounting smooths stories over time, and the sharpest seam is the migrating one-off. What one quarter calls a one-time logistics disruption, the next quarter absorbs into operating costs without ceremony. A pipeline aligning quarters by surface text will drape the expense into opposite categories without raising its hand; a corpus has no referee for when a category moved, because usually nobody annotated the move.
Here is where earnings text is cruel to machines in a way news is not: the document’s job is continuity of narrative across periods, and the machine’s job is alignment across the same periods. The quarrel has no referee, and item-by-item footnotes win the match only for readers willing to do what models are bad at — keep a ledger of what changed meaning rather than wording.
Table trouble, consequent and neglected
Real filings run their numbers through tables whose units live in the header, whose comparatives live two pages back and whose footnotes reinterpret a column mid-document. Language models approximate that structure badly; extract a 3.2 without its unit and you hold a number wearing borrowed clothes. Structured parsers do better on the grid and worse on the prose around it, so production pipelines fuse both and then inherit a combined list of ways to misalign — which responsible teams publish and most marketing omits.
Six questions that keep the claims honest
When a tool announces that its artificial intelligence reads earnings reports, ask six things. Does it quote the passage it scored? Does it define its hedge counts, or keep them an industrial secret? Does it keep adjusted metrics separate from unadjusted ones, tagging the difference? Does its sentiment score run against the calendar or against the company’s own history? Does it remember when categories migrated between periods? And does it ever show a failure of its own unprompted? A polite silence on one is survivable; politeness cannot substitute for answers on all six.
Failures catalogued like this are the closest a review of method comes to balance. The machine reads quarterly prose with unusual patience and unusual blindness, and recording both — with passages and numbers — is what this shelf is for.