How does era sensitivity in legal cases compound with context length failures?

This explores whether two separate weaknesses — LLMs doing worse on older legal precedent, and reasoning breaking down as inputs get longer — share a common root and stack on top of each other. The corpus suggests they aren't independent failures at all: both trace back to the same thing, which is how unevenly the training data is distributed.

Start with era sensitivity. On a Supreme Court overruling benchmark, models reliably do worse on historical cases than modern ones, and the diagnosed cause isn't the law being harder — it's that recent cases flood the training corpus while older precedent is thinly represented, leaving the model with shallower internal representations of it Why do language models struggle with historical legal cases?. That's not a quirk of legal text. The same fingerprint shows up in temporal reasoning generally: models stay competent on short, structured time questions but start generating impossible timelines in long open-ended contexts, and that breakdown 'tracks training data distribution' as the model falls back on frequency heuristics instead of actually reasoning Why do language models fail at temporal reasoning in complex tasks?. Both failures are the model leaning on what was common in training when the going gets hard.

Now the context-length half — and here's the part most people underestimate. Reasoning accuracy doesn't just degrade near the context window limit; it drops from 92% to 68% with only 3,000 tokens of padding, far below capacity, and chain-of-thought prompting doesn't rescue it Does reasoning ability actually degrade with longer inputs?. A complementary view reframes the bottleneck as not memory but the compute needed to consolidate everything in the window into usable internal state Is long-context bottleneck really about memory or compute?. So a long historical case file is a double tax: the model already has a thin grip on the era, and the length itself is eroding whatever reasoning it could muster.

The compounding mechanism becomes clearer through a third lens — failures are driven by instance-level unfamiliarity, not raw complexity. Models succeed on any reasoning chain when they've seen similar instances and fail at novelty boundaries, because they fit instance patterns rather than general algorithms Do language models fail at reasoning due to complexity or novelty?. A historical case is precisely a low-familiarity instance, and a long document pushes more of that unfamiliar material through exactly the conditions where reasoning is most fragile. The two weaknesses don't add — the era gap makes the content novel, and length is the multiplier on novel content.

What you didn't know you wanted to know: the corpus points to a defense that sidesteps both. A multilingual RAG system built for noisy, drifting historical newspapers wins not by reasoning harder but by aggressively expanding retrieval while forcing the model to refuse any answer it can't ground in evidence — trading coverage for integrity exactly where source quality is degraded Can RAG systems refuse to answer without reliable evidence?. The catch is that long context alone won't substitute for this: long-context models match retrieval on semantic tasks but fail on structured, relational queries, so stuffing the whole case file into the window is the worst of both worlds Can long-context LLMs replace retrieval-augmented generation systems?. The escape hatch isn't a bigger window — it's grounding plus the discipline to abstain when the era is thin and the document is long.