Sources and Methods

How the CaseDiver corpus is built and searched

This page is the long account of where the data comes from, how it is kept current, how citations are found and judged, how meaning-based search works, and how the databases fit together. It is not a product pitch. For licensed wholesale access, see Bulk Data.

Public sources, no vendor lock-in

CaseDiver represents and warrants that the entire CaseDiver database can be used without violating any licenses, copyrights, or terms of service. The corpus was assembled without accepting any license, terms of service, or other restriction that would constrain how CaseDiver uses the data, or how CaseDiver customers are able to use the data.

CaseDiver does not subscribe to a commercial caselaw feed, does not scrape a paid research platform, and does not ingest anyone else's headnotes or citator. The inputs are public records and public government publications.

All caselaw analysis in CaseDiver was created by CaseDiver. Nothing was taken from other research services. Westlaw headnotes were not used. Lexis headnotes were not used. Shepard’s was not used.

Where the text comes from

CourtListener quarterly downloads

The historical backbone is the quarterly bulk downloads published by CourtListener. Those dumps are available without restriction. They supply the mass of older federal and state opinions that no live crawler could reasonably reconstruct from scratch — on the order of 9.9 million opinions once normalized into CaseDiver's schema.

Direct harvest of court websites

CaseDiver harvesters work around the clock. They visit more than 200 individual court websites — supreme courts, intermediate appellate courts, and federal courts that publish their own slip opinions — and pull newly posted decisions as they appear. Courts do not publish on a single schedule. Some post in the morning, some after the close of business, some in batches at the end of the week. Continuous harvest is how yesterday's opinion is in the index today rather than in next quarter's dump.

Public RSS feeds

Where a court or reporting service exposes a public RSS (or Atom) feed of new opinions, CaseDiver subscribes. Feeds are a second, independent signal next to HTML harvest: if a site's listing page is slow to update but the feed is current, the feed still wins.

GovInfo

GovInfo (govinfo.gov) is the Government Publishing Office's public access system. It holds federal statutes, the Code of Federal Regulations, and a large body of federal caselaw. CaseDiver uses it as the authoritative public source for the United States Code, the C.F.R., and complementary federal opinion text — not as a licensed commercial product, but as the government's own publication of the law.

Coverage of the caselaw layer is U.S. Supreme Court, federal courts of appeals, federal district courts, and the appellate and supreme courts of all fifty states.

United States Code, federal rules, and the C.F.R.

CaseDiver is wired directly into twenty-five years of the United States Code. The code is fully accessible as first-class text, not a citation string pointing somewhere else. It is cross-indexed in both directions: every case that cites a given code section, and every code section that is cited by a given case. Opening 42 U.S.C. § 1983 should show the statute and the graph of opinions that treat it; opening an opinion should show the sections it relies on.

The same design covers the federal rules: the Federal Rules of Civil Procedure, Criminal Procedure, Evidence, and Appellate Procedure, stored as rules and subdivisions the way lawyers actually cite them (Rule 12(b)(6), Rule 56, Rule 403), and linked to the opinions that invoke them.

The same cited-to / cited-by architecture is being applied to twenty-five years of the Code of Federal Regulations. You cannot see the C.F.R. layer in the public app yet — it is being wired now — but it is the same graph, the same stores, and the same retrieval pattern as the U.S. Code.

How vector search and embeddings work

Keyword search looks for the words you typed. Vector search looks for the meaning of what you typed, even when the opinion never used those words. The two are complementary; CaseDiver runs both.

What an embedding is

An embedding model reads a passage of text and writes it down as a list of numbers — a vector. CaseDiver uses Qwen3-Embedding-8B at 1024 dimensions. That means each opinion (and each analysis card) becomes a point in a 1024-dimensional space. Passages about the same legal idea land near each other, even if one says "unlawful detainer" and the other says "eviction for nonpayment of rent."

The model was trained to make that geometry true for language in general. CaseDiver then embeds the corpus with it, stores the vectors, and at query time embeds the user's question with the same model so the question becomes a point in the same space.

Cosine similarity, in plain terms

Distance between two vectors can be measured many ways. CaseDiver uses cosine similarity: it cares about the angle between two vectors, not how long each vector is. Imagine two arrows starting from the origin. If they point the same direction, the angle is near zero and cosine similarity is near 1 — a strong match. If they point in unrelated directions, the angle opens toward 90 degrees and cosine similarity falls toward 0. If they pointed opposite ways (rare in this embedding space), cosine would go negative.

Length is ignored on purpose. A 200-page opinion and a two-paragraph holding can still be "about the same thing." Cosine asks whether they point the same way, not whether one arrow is longer because the opinion was wordy.

A concrete example

Suppose you search: "Can a landlord keep a security deposit for ordinary wear and tear?" A keyword engine looks for "landlord," "security deposit," and "wear and tear." It will miss a well-reasoned opinion that talks only of "the statutory duty to return prepaid rent minus documented damage beyond deterioration from reasonable use," because those tokens never appear together in your query. A vector engine embeds your question, then finds opinion and analysis vectors whose arrows point the same way — often the cases you actually wanted.

A second example: search "fruit." In cosine space, "apple" and "orange" sit close to "fruit." "Car" does not. Legal language works the same way. "Qualified immunity" sits near "clearly established constitutional right" and near "Harlow v. Fitzgerald," even when the query never named the case. "Maritime lien for necessaries" does not sit near a consumer security-deposit dispute, even if both opinions contain the word "lien."

Two vector sets, not one

CaseDiver embeds each opinion's body, and separately embeds CaseDiver's own analysis of that opinion — issue, holding, posture, disposition. A question phrased as a lawyer would ask it matches the analysis set, because that text is already written in the language of issues and holdings. The body set catches what the analysis omitted. Search uses both; recall is the union.

Why Qwen 8B at 1024 dimensions

Qwen3-Embedding-8B at 1024 dimensions is the state of the art for a corpus this large. Smaller models are cheaper and miss distinctions that matter in law (distinguishing "dismissed with prejudice" from "dismissed as moot"). Much larger models and much wider vectors improve laboratory scores a little and explode index size, memory, and latency when you must store and search on the order of 470,000 vectors. Eight billion parameters and 1024 dimensions is the point on the curve that maximizes accuracy without giving up the speed a live research tool needs. Vectors are stored as pgvector half-precision values and retrieved with a binary-quantized HNSW index: a fast Hamming first pass, then a full-precision rerank of the shortlist.

Why vector search plus boolean is stronger than either alone

Boolean (and BM25 full-text) search is exact. You can require a reporter citation, a docket pattern, a statute section, a date range, a court, a phrase in quotes, or AND / OR / NOT logic. That is how you know you have this case, or every opinion that quotes Rule 56 in the Tenth Circuit after 2018. Vector search cannot give you that guarantee. It gives you neighbors in meaning-space, which is a different promise.

Used together they outperform either used alone, and they outperform research systems that are only a citator plus keywords. A typical CaseDiver pattern:

• Constrain with boolean and structured filters: jurisdiction, date, publication status, a statute or rule, a phrase that must appear.
• Rank and expand with cosine: among those candidates — or across the unconstrained corpus when you do not yet know the magic words — surface opinions whose meaning matches the question.
• Walk the citation graph on the shortlist: what those cases cite, and what later cites them, including treatment from GoodLaw.

Keyword-only tools fail when the doctrine has many names. Vector-only tools fail when you need a specific reporter cite or must exclude a statute. The combination is what makes "find me the cases that actually answer this question, in this court, still good law" a single workflow instead of three products.

GoodLaw, and why it is not Shepard’s or Westlaw

Shepard’s and the Westlaw citator were built by generations of human editors, later assisted by older automated systems, reading what they could of the reporters. They are extraordinary artifacts. They are also bounded by what humans and earlier software could finish.

CaseDiver’s proprietary GoodLaw system takes a different approach. It is trained to recognize the full range of legal citations — full case cites, short forms, id. and supra, statutes, rules, and the ugly in-text variants courts actually write — and it is run against every citation in the corpus, on the order of ten million opinions and 182 million in-text cites. A highly proprietary prompting harness then drives an AI pass that literally examines those citations in context: what the citing court is doing with the authority, not merely that a string appeared.

That scale of examination is not something a human editorial workforce can complete across the live federal and state corpus, and it is not something earlier AI systems could do reliably. The result is a citator graph CaseDiver built itself, not a licensed extract of Shepard’s or KeyCite, and not a copy of anyone's headnotes.

GoodLaw started from the open-source eyecite application (Free Law Project) and was greatly enhanced in-house: more citation types, tighter reporter resolution, short-form chains, and a reconciliation step against CaseDiver’s own reporter-key index so a raw match becomes a resolved graph edge.

Deterministic analysis plus a probabilistic layer

Two different kinds of machine are stacked on purpose. Confusing them is how systems become either brittle or untrustworthy.

Deterministic (GoodLaw)

A deterministic procedure returns the same output for the same input, every time. Given one opinion text, GoodLaw will find the same citation spans, the same reporter keys, and the same short-form resolutions on Monday that it found on Friday. That is what makes the graph reproducible, auditable, and safe to rebuild. It increases reliability (you can re-run and get the same edges), accuracy on the citation types the grammar knows, and true duplication of results across environments — the same corpus plus the same scanner yields the same database.

The cost of determinism is blindness to what the grammar does not encode. Courts invent cite forms. OCR garbles reporters. A statute may be referred to only by popular name. A purely deterministic scanner will systematically miss those.

Probabilistic (the LLM pass)

A probabilistic model does not guarantee the same tokens twice, but it can notice a citation a grammar never defined. The LLM pass is run as a completeness layer: it is asked to find citations GoodLaw did not mark, in the same opinion, under a harness designed to suppress invention. Finds are stored as a distinct source so a researcher can see which edges came from the parser and which from the model.

Stacked this way, determinism is the backbone and probability is the net. You get repeatable structure for everything the grammar knows, and you still catch what it does not, instead of choosing one philosophy and living with its failure mode.

CiteDiver

CiteDiver is the product that applies this stack to a brief or memorandum you upload. It does not guess whether a cite "looks real." It resolves each citation against the CaseDiver corpus using GoodLaw — the same deterministic recognizer that built the graph — and then uses the LLM pass to catch cites the grammar missed in your document, the same way it does inside the corpus. Each cite is reported as confirmed, unresolved, or carrying treatment that should worry a filer (for example a later opinion that overrules or questions the authority).

Because the corpus itself was built without Westlaw or Lexis content, CiteDiver is not a wrapper around Shepard’s. It is CaseDiver’s own authority file, CaseDiver’s own extractor, and CaseDiver’s own treatment analysis, pointed at the document in front of you.

The databases and how they work together

CaseDiver is not one table and not one database. Retrieval by citation, by name, by keyword, by meaning, and by statute are different physical problems. They are stored separately and queried in sequence, never mashed into a single scan.

Three PostgreSQL databases

Corpus (303 GB) is the store of record for the law itself: 9.9 million rows in the opinion table (majority, concurrence, and dissent each addressable); 78 million resolved opinion-to-opinion edges; 182 million in-text citations with character offsets and pin cites; 18 million reporter citations with a normalized reporter key; authority citations that point at statutes and rules; AI analysis rows kept physically apart from court-authored text; and the embedding table (1024-dimension half-precision vectors with a binary HNSW index). Statute sections and court rules live here as well, versioned so a 2019 cite can still resolve to 2019 text.

Search holds two lookup indexes and almost nothing else: a citation index of 15.5 million rows keyed on a normalized reporter key (so "410 U.S. 113" is one indexed hit, not a scan of nine million opinions), and a name index of 9.7 million rows with trigram indexes on case name and party names so "Roe v. Wade" still works if you type "roe wade" or a misspelling.

Control holds users, sessions, configuration, usage logs, and the job queue that drives harvest, embed, and analysis. It does not hold the law. Application queries never join control to corpus in one statement; IDs are resolved in one store and used in the next.

OpenSearch (Apache Lucene)

A 133 GB inverted index over 9.8 million opinion documents supports boolean, phrase, and proximity queries with BM25 ranking and field-level boosts on body, name, and syllabus. This is the keyword half of "vector plus boolean."

Object storage

Official PDFs and harvested source files live in object storage. The databases store text, metadata, and vectors; the PDF is fetched when a researcher wants the court's original pagination.

A query, end to end

A researcher types a question and maybe a court or date filter. The citation index answers if the query is a cite. The name index answers if it is a case name. The Lucene index answers boolean and phrase constraints. The vector index answers meaning, across body and analysis embeddings, with cosine ranking. Hits are opinions in the corpus store. From an opinion, the citation graph walks cited and citing cases; from a statute or rule, the authority tables walk the same graph in the other direction. New opinions arriving from harvest are normalized, GoodLaw-scanned, LLM-completed, embedded, and pushed to every store so the pieces stay one corpus rather than five stale copies.

Around the clock

Harvest, citation scan, embedding, and analysis are jobs, not a quarterly ritual. Courts publish when they publish. Harvesters keep walking the 200-plus sites, the RSS feeds, and GovInfo. Each new opinion is folded into the same tables and the same indexes described above. The public research UI will grow as the U.S. Code, federal rules, and C.F.R. wiring is finished; the method does not change when a new layer becomes visible.

Short answers

Westlaw, Lexis, Shepard’s, and KeyCite are trademarks of their respective owners. CaseDiver is not affiliated with, endorsed by, or sponsored by them. Nominative references above describe what CaseDiver does not use.