Hybrid Scoring
A tri-phase composite system combining deterministic verification with qualitative judgment to ensure legal reasoning integrity.
The Problem with Single-Method Scoring
Purely rule-based scoring misses the nuance of legal arguments, while purely LLM-based judging can be subjective and brittle. LegalChain solves this by using a Composite Scoring model that captures both structure and substance.
Structure
Consistency
LLM Judge
Architecture of Valuation
PHASE 1: STRUCTURAL PRESENCE (10%)
Ensures the model produced all IRAC components—Issue, Rule, Application, and Conclusion. This is a binary gate: either the skeleton exists, or the score is zeroed for this phase.
PHASE 2: CHAIN CONSISTENCY (40%)
Deterministic fidelity checks (CC1-CC5) that compare the S6 synthesis against prior step outputs (S1-S4). Catches internal contradictions where the model's conclusion disagrees with its own prior reasoning.
See Consistency Metrics →PHASE 3: LLM JUDGE (50%)
Qualitative evaluation of legal reasoning merit. A judge model examines the persuasiveness and accuracy of the rule application using established Judging Policies.
Handling Judge Unavailability
In exceptional circumstances where the LLM Judge is unavailable, the system can shift priority to deterministic metrics. Important: Runs executed in resilient mode are flagged in scoring details. Full benchmark validity requires judge scoring; resilient mode provides diagnostic data only.