AI summaries

What a summary is (and isn’t)

For each material section change, Billy generates a structured summary: headline, plain-English explanation, optional numeric-change chips, and evidence quotes. Each card shows a change-type chip with a compact size meter (edit volume) and a small confidence orb (AI trust signal — not legal impact). Hover or tap either for the key and details. Bill-level rollups combine section summaries into a short narrative. Minor edits (very few words changed) skip summarization and are marked as such.

Summaries help you find what matters. They are not a substitute for reading Redline mode or the official PDF.

Click to enlarge

How evidence grounding works

Peer-reviewed evaluation found Lexis+ AI hallucinating on ~17% of queries and Westlaw on ~33% (Magesh et al., J. Empirical Legal Stud. 2025). Retrieval grounding alone is not legal-grade — which is why Billy does not stop at RAG.

Every claim in a summary is backed by the bill text itself. Here’s how that works:

1. The model is prompted with aligned before/after section text and word-diff ops, and instructed to describe only changes evidenced in the diff — no speculation about legislative intent.
2. Claims return verbatim quotes that should justify each assertion (a few words up to a few sentences; sparse quotes allowed when a change is scattered).
3. Billy locates each quote in the canonical text (exact match, then fuzzy within the section) and stores resolved character ranges.
4. Those ranges resolve through a source map to page/bounding boxes on the original PDF so the evidence panel can highlight them.

Semantic audit (second pass)

Locating a quote proves the words appear in the bill — not that the claim accurately describes the change. After quote-anchoring, Billy runs a semantic entailment check against the diff: each claim is labeled supported, partially_supported, or not_supported, with a short rationale. Audit badges surface in the verification ledger; counsel check-offs always win visually over the machine verdict.

Click to enlarge

Verification ledger

On a tracked bill (and from the Diff Viewer), open Review to see every summary claim in one place: evidence status, AI audit badge, and your Verify / Reject check-off with optional notes. When you are satisfied, sign off the bill — Billy records who signed and how many claims were verified. Export the ledger as DOCX or PDF for the file.

Click to enlarge

Affirm, correct, dispute

• Affirm — you reviewed the claim and agree. Badge shows who reviewed and when.
• Correct — provide replacement summary text (and optionally corrected evidence). The editor starts from the AI text.
• Dispute — flag as wrong with required rationale; no replacement text required.

Revisions layer over the original; the AI text is never mutated and stays one click away. Reviews are workspace-private client work product — they do not leak across organizations. They emit feed events and appear in exports.

When a summary is regenerated

Summaries are globally cached per version pair, prompt version, model, and normalizer versions. Reprocessing with a newer normalizer (or prompt) can supersede prior summaries — Billy does not delete the old rows. Your affirm/correct/dispute stays attached to the text you actually reviewed, with an “AI summary updated since review” indicator when a newer generation exists.

Returning to the same diff shows byte-identical analysis until inputs change. An affirmed summary remains the system of record for that review.

When summaries compute

Structural diffs run for every consecutive version pair (no LLM cost). LLM summarization runs eagerly for tracked bills and bills at/above suggestion threshold; other pairs compute lazily on first view and then memoize globally so orgs share the same analysis.