Frozen Context Spec

Defining the "Atomic ResearchPack" through byte-for-byte reproducibility and frozen constant extraction.

(Content synced from Methodology B)

The Crisis of Context

Most RAG benchmarks treat "retrieval" as an opaque process. If you run the benchmark today, you might get different snippets than you did yesterday due to index shifts, embedding updates, or nondeterministic re-ranking. This makes scientific comparison impossible: Did the model improve, or did it just get lucky with better context?

The Fix: The Frozen Context Spec

LegalChain treats the context window itself as a normative scientific artifact. The construction of a ResearchPack (RP) is governed by a strict, "frozen" specification (Step 4A) that guarantees byte-for-byte reproducibility.

Core Principles

  • No Runtime Retrieval: Context is built once at build time, hashed, and sealed.
  • Deterministic Trimming: Anchors > 80k chars are trimmed using a strict "Head-Tail" strategy (first 40k + last 40k), ensuring widely cited intro/conclusion text is preserved.
  • Frozen Constants: All "magic numbers" (e.g., snippet window = +/- 80 chars, Top-K = 12) are hardcoded in the rp_constants.py spec.
  • Zero Randomness: All tie-breaking (e.g., sorting citations with identical relevance) uses lexicographical keys (e.g., citation string).

The "RPv1.1" Scoring Formula

To rank thousands of potential citations down to a "Top 12" context window, we use a deterministic scoring formula. This formula is frozen for the duration of the benchmark version.

# Deterministic Relevance Score (Frozen)
Score = E + W_t(label) + 0.2*log(freq) + W_pos + W_density - 0.5*Overruled

E = Edge exists (1.0|0.0)
W_t = Treatment weight (Followed=1.0, etc.)
W_pos = Positional boost (Intro > Body)
W_density = "Holding" keyword density

CAP Integration (RPv1.1)

For federal cases (Caselaw Access Project) where high-quality Shepard's signals (E) are missing, the formula automatically adapts to rely on Semantic Density (W_density) and Positional Authority (W_pos). This ensures a unified standard for both SCOTUS and Federal context.

Auditability

Every generated ResearchPack includes a rp_manifest.json containing:

  • Git SHA of the builder code
  • Exact values of all constants used
  • SHA-256 hashes of all input datasets
  • Coverage statistics (how many candidates were found vs selected)

This allows any researcher to verify that the context window they are evaluating was built according to the strict v1.1 standard.