Product

·

Why LLMs can't do legal research

The biggest misconception in legal AI today is that LLMs can reliably perform legal research. No matter if you plug in an external database or not.

It sounds intuitive. If an AI can read legislation, case law, and legal literature, surely it should be able to answer legal questions correctly.

Unfortunately, that's not how legal research works.

The problem isn't reading. It's finding.

Legal research is fundamentally a retrieval problem before it's ever a reasoning problem.

To answer a legal question correctly, you first need to identify which authorities matter. Only then can you reason about them. This is exactly where today's frontier language models struggle.

The 'Legal Research Benchmark' by vals.ai shows that even the absolute frontier of models will give you the correct sources in only 50% or less amount of times. That means it’s a coin flip if you got the right ones.

In a high-stakes game like legal services, you don’t want to rely on coin flips. It defeats all of the promised productivity gain.

Why general LLMs struggle with law

Many people assume the problem is hallucination. That's only part of it.

The deeper issue is that legal references aren't naturally organized in a way that's easy for a general-purpose language model to navigate. Relevant legal references often:

  • aren't publicly available on the open web,

  • depend on jurisdiction-specific databases,

  • require understanding relationships between statutes, case law, commentary, and procedural history,

  • and only become relevant once the precise legal issue has been identified.

Finding the right legal reference is rarely done by keyword matching. Lawyers also shouldn’t search for documents but rather for the argument that changes the analysis.

"I'll just connect ChatGPT to my old legal database."

This is probably the most common proposed solution.

Just connect your favorite LLM to an incumbent legal database. Problem solved.

Not quite. Giving an LLM access to millions of legal documents doesn't automatically make it good at legal research.

Imagine dropping someone into one of the world's largest law libraries. Every book is available. Every journal is available. Every judgment is available. But there's no librarian, no catalog, and no one who understands exactly what legal issue you're trying to solve.

The information exists. But finding the right legal reference efficiently is still incredibly difficult. Adding more documents just increases the search space without solving retrieval quality.

"Google will bring you back 100,000 answers. A librarian will bring you back the right one."

The librarian: What good legal research AI actually looks like

The future of legal AI isn't going to be replacing legal research with an LLM. We need to build systems where legal retrieval and legal reasoning work together. That means systems specifically designed to:

  • understand legal context rather than keywords,

  • identify the most relevant authorities across legislation, case law, and legal literature,

  • reason over those authorities,

  • and provide answers that can be verified back to their original sources and throughout time.

Only then does the language model become genuinely useful. The LLM isn't the legal researcher but the communicator. The legal research engine is the librarian and determines whether the answer can be trusted.

If you want to save time in legal research, find the best librarian and not the biggest library.

Why we decided to be librarians, before building a library.

There are enough libraries, but no good librarian.

Pandektes was built on a simple belief: legal research is much harder than most people think.

Not because lawyers aren't good at it. And not because the law itself is unusually complicated. It's hard because the problem has two properties that, together, make it unforgiving.

First, the universe of relevant legal information is enormous. Statutes, case law, commentary, regulations, guidance - spread across jurisdictions, languages, publishers, and decades. Much of it isn't openly available, and almost none of it is organized around the question you're asking today.

Second, legal research has an unusual standard for success. Being right most of the time isn't enough. Miss one highly important reference, overlook one important judgment, or cite the wrong source, and trust disappears. Small details are often the deciding factor.

That's why we've always approached legal AI differently.

Many companies start with a language model and ask, "What can we build with this?"

We started with the research problem itself and asked, "What would it take to solve this properly?"

The answer turned out to be much harder and much more interesting than we imagined.

Legal professionals spend countless hours searching through old databases or libraries in our analogy. It's often the most time-consuming part of their day, yet the tools they've relied on for decades haven't changed much. Too often, they feel like asking a librarian who eventually forgot that the job wasn't to point at shelves. It was to help people find the right book quickly, confidently, and every single time.

Lawyers deserve better than that.

We think it's possible to build a system that doesn't just search faster, but actually understands the question, reasons about the law, and consistently surfaces the authorities that matter.

That's a problem worth spending decades on.

See it for yourself.

A short walkthrough on your own questions — and where every answer comes from.

See it for yourself.

A short walkthrough on your own questions — and where every answer comes from.