Context Rot & Knowledge Decay — Research Round 5

Memory Commons reference library · 7 July 2026.

Verification status: INTERRUPTED — treat every claim below as source-attributed but UNVERIFIED. This round’s adversarial verifiers aborted mid-run on credit exhaustion. The pipeline bucketed all 25 extracted claims as “refuted”, but their votes are all 0-0 — meaning verification never executed, not that they were refuted. The sources are legitimate primary/standards references and the claims match well-established literature, but by this project’s honesty convention (rounds 1–4 carried real 3-0 votes) they are not yet canonical. Re-verify before citing in any load-bearing decision. Ironically apt: the research pipeline itself hit a failure mode, and the honest record of that is the point.

Why this matters to Memory Commons

A project whose purpose is to defeat civilizational forgetting must not itself forget between sessions. Two distinct threat models, both real:

• LLM context rot — degradation within and across AI sessions: models lose accuracy as context grows, stale memories get reused, compaction drops facts, and details drift. This is the session-to-session failure mode.
• Human/institutional knowledge decay — degradation over years: documentation drifts from reality, links rot, decisions decay into folklore, and knowledge leaves when maintainers do. This is the decade-to-decade failure mode.

The controls established in response are in governance/context-integrity.md and decision record decisions/0008.

Claims by section (source-attributed, UNVERIFIED this round)

A · LLM context rot (the session failure mode)

• Chroma “Context Rot” (Jul 2025, 18 models incl. Claude 4, GPT-4.1, Gemini 2.5): performance degrades as input length grows even with the task held constant; lower needle–question semantic similarity steepens the decline; distractors compound harm; models did better on a shuffled haystack than a coherent one — organization, not just presence, drives reliability. (research.trychroma.com/context-rot; chroma-core/context-rot)
• “Lost in the Middle” (Liu et al., TACL 2024): U-shaped position effect — info at the start or end is used well, the middle is lost; GPT-3.5 dropped >20 points mid-context, below closed-book in the worst case; bigger context windows alone did not fix it. (aclanthology.org/2024.tacl-1.9)
• NoLiMa (2025): with lexical shortcuts removed, 11/13 “128K” models fall below 50% of short-context baseline by 32K tokens; even GPT-4o fell 99.3%→69.7%. (arXiv:2502.05167)
• RULER (NVIDIA): vanilla needle-in-a-haystack overstates ability; effective context is often half the claimed window. (github.com/NVIDIA/RULER)
• LongMemEval: ~30% accuracy drop over sustained multi-session memory; a focused prompt beats the full ~113k-token history — direct evidence that just-in-time retrieval of minimal relevant context beats preloading. (arXiv:2410.10813)

B · Context-engineering controls (unverified this round)

External memory (files as memory, plan documents), just-in-time retrieval over preloading, disciplined compaction, and durable project docs (CLAUDE.md/AGENTS.md conventions, decision logs). (To be re-sourced against Anthropic context-engineering guidance in a clean round.)

C · Documentation decay over years (unverified this round)

Link rot (Pew 2024: ~38% of 2013 pages gone by 2023; Zittrain SCOTUS ~49%), doc drift/staleness, ADRs (Nygard 2011) as the decision-capture control, docs-as-code single-source-of-truth, bus-factor loss when maintainers leave.

D · Archival-science anti-rot controls (unverified this round)

OAIS (ISO 14721): SIP/AIP/DIP, fixity, preservation planning. PREMIS preservation metadata. NDSA Levels of Digital Preservation (storage / integrity-fixity / control / metadata / content). LOCKSS (“lots of copies keep stuff safe”) + routine checksum audits. These standards frame the controls catalog even though this round could not verify their specifics.

Action taken regardless of verification gap

The controls in governance/context-integrity.md and decision 0008 were implemented immediately — frozen release snapshots, a CI fixity gate, a canonical source hierarchy, decision records, and a session-start protocol. These rest on the project’s own engineering (Merkle fixity was already in place) and standard practice, not on this round’s unverified claims. Round 6 should re-run verification cleanly and confirm sections A–D.