A legal answer is different from a normal chatbot answer.

If a recipe bot guesses, dinner may be worse. If a legal bot guesses, the mistake can move into a court filing, a landlord dispute, a benefits appeal, a workplace problem, a family decision, or a bill someone cannot easily fix.

That is why the first practical question is: who verifies the law before the answer becomes someone’s next step?

Courts are already being forced to answer that question. The Florida Supreme Court amended filing rules to address AI use in court filings. A Princeton Center for Information Technology Policy article by Patty Liu, Dominik Stammbach, and Peter Henderson says their tracking, combined with other efforts, has identified more than 1,000 filings containing hallucinated citations. In the research preprint The New Pro Se, Or Cohen-Sasson argues that since generative AI tools became widely available, federal civil courts have seen more people representing themselves, along with questions about complaint text, outcomes, and court workload.

For someone who has never been able to afford a first consultation, more access matters. More people may be able to draft a question, understand a form, or prepare for a legal conversation without paying for every first step. That part is real. Legal help is often too expensive, too slow, or too hard to reach.

But access and correctness are not the same thing.

A person using AI for a legal problem still needs to know what source the answer used, whether the cited case or rule exists, whether the answer fits the right state or court, and whether a real person has checked it. A court still needs to know whether a filing rests on real law. A lawyer still signs the work. A self-represented person may not even know what to verify.

Joseph Kwon of Joe Kwon Law, who runs what he described as an AI-native immigration practice, pointed to immigration as a clear test case because it has one of the country’s largest self-represented populations. His warning was simple: the access gain from chatbots is real, but so is the failure mode when a confident wrong answer appears in an interview or request for evidence and becomes expensive or impossible to fix.

The new legal skill may be less impressive-looking than the demo: show the sources, check the citations, name the human owner, and keep a path to repair.

Josh Elberg, founder of Palavir, made the verification point even more bluntly in comments shared with The Fast Now: “AI accelerates the analysis but moves the entire burden onto verification. In legal work a hallucinated citation or an inflated damages figure is not a bug, it is malpractice.”

Legal AI needs a visible check before its clean answers are treated like legal reality. That is a guardrail problem, not a reason to avoid the tools.

If a tool helps someone understand a form, good. If it drafts a letter, useful. If it points someone toward questions for a lawyer, helpful. But when the answer could affect housing, custody, debt, immigration, benefits, employment, or a court record, the question changes.

Who checked the law?

Who owns the answer?

Who can fix it if the machine was wrong?

The fast now is where a private chatbot answer starts touching public systems. Legal AI may open the first door for more people. The next door still needs a real check.

What to watch

  • Do courts require lawyers or self-represented filers to certify AI-assisted citations?
  • Do legal tools show the exact source text behind an answer?
  • Can a user tell whether the answer applies to the right state, court, or deadline?
  • Is there a named person or office that can correct the mistake?
  • Does easier access create more work for clerks, legal-aid groups, and judges?

Source note

Sources include Florida Bar coverage of Florida Supreme Court rule changes, Princeton Center for Information Technology Policy citation-verification research, the research preprint The New Pro Se, NCSC access-to-justice programming, HMCTS public AI adoption posture, and source comments shared with The Fast Now by Joseph Kwon of Joe Kwon Law and Josh Elberg, founder of Palavir.