Adversarial verification framework for AI-generated legal content: fact-checking, citation validation, hallucination detection, and distribution readiness scoring.
npx clawhub@latest install legal-red-teamLegal Red Team is a production-ready adversarial verification framework for AI-generated legal documents. It systematically checks factual accuracy, validates legal citations against official sources, detects known hallucination patterns, verifies arithmetic, and scores documents for distribution readiness — all across a structured six-category methodology. This skill does not constitute legal advice and is intended to supplement, not replace, qualified professional legal review.
Every document is assessed across six structured categories: factual accuracy, legal authority citations, arithmetic validation, source verification, speculation detection, and disclaimer adequacy. Each category has defined red flags and checks, ensuring no common failure mode is missed.
The skill targets five recurring AI hallucination patterns in legal content: plausible-but-wrong article numbers, confidently incorrect dates, guidance mischaracterized as binding law, outdated legal references, and arithmetic errors in timeline calculations. Each pattern has a defined detection technique.
Findings are classified into four severity levels — CRITICAL, HIGH, MODERATE, and LOW — with clear definitions, examples, and required actions at each level. CRITICAL issues must be resolved before any distribution.
The legal_quality_scorer.py tool produces a composite 1–5 score with per-category breakdown. Documents scoring below 4/5 must not be distributed; the pre-distribution gate workflow enforces zero CRITICAL issues and complete disclaimers.
Two Python scripts — legal_fact_checker.py and legal_quality_scorer.py — support text or file input, JSON output, verbose mode, and saved reports. They serve as first-pass scanning tools, designed to feed into manual adversarial review.
Three ready-to-run workflows cover full adversarial review, quick citation checks, and pre-distribution gating. Each workflow includes a validation step to confirm completion criteria have been met.
Run legal_fact_checker.py to flag all citations and dates, verify each against EUR-Lex or eCFR, then run legal_quality_scorer.py to confirm a 4/5 or higher score and zero CRITICAL findings before sending to clients or staff.
Use Workflow 2 (Quick Citation Check) to extract every legal citation from the document and verify each against the relevant official source — catching invented article numbers or misquoted provisions before they reach a legal team.
Integrate legal_quality_scorer.py as an automated gate in a document generation pipeline. Any document scoring below 4/5 is held for human review; only documents meeting the threshold and disclaimer requirements are passed downstream.
Apply the six-step methodology with an adversarial mindset: mark every factual assertion and number, verify dates against statutory text, flag speculation presented as certainty, and produce a severity-classified findings report for the drafting team.
npx clawhub@latest install legal-red-teamnpx clawhub@latest install legal-red-teamLog in to write a review
No reviews yet. Be the first to share your experience!