1

Amid the expansion of generative AI legal services, cases where AI-generated false judicial precedents are cited in legal briefs have occurred in succession, fueling mounting concerns over the reliability and verification systems of AI legal tools.

According to the legal community on the 18th, the adoption of AI legal services began to spread in earnest across the domestic legal tech industry starting in 2024, as AI-based legal search and document analysis services emerged one after another.

Notably, L-Box introduced its legal research service "L-Box AI" in April of that year, while Law&Company launched its attorney assistance service "Super Lawyer" in July. The National Court Administration of the Supreme Court also initiated a pilot operation in February this year of a generative AI system designed to assist judges and court personnel with trial proceedings.

The problem lies in the fact that inaccurate legal information is impacting court proceedings, with unverified AI-generated information being utilized in trials and resulting in fabricated precedents being cited in legal briefs.

In fact, the Patent Court pointed out in a ruling on June 11 that an attorney cited and submitted eight AI-generated false precedents in a legal brief without prior verification. The Suwon District Court also stated in a civil ruling on May 28 that precedents presented by a litigating party appeared to be fabricated cases concocted through AI.

AI errors were thus left unfiltered not only in pro se litigation, but also in cases represented by licensed attorneys. A criminal defense attorney who previously served as a prosecutor explained, "Even if AI assists with precedent searches and drafting briefs, attorneys must separately verify whether cited precedents actually exist and whether the legal principles are applicable to the case at hand."

Cases where AI-generated false precedents were cited in actual litigation were also shared during judicial training sessions. During a judicial seminar held by the Judicial Research and Training Institute on the 4th of last month under the theme of "Measures to Respond to the Citation of AI-Generated False Statutes and Precedents," an instance was introduced where 39 out of 40 precedents cited in a brief submitted by a litigating party turned out to be completely non-existent fabricated cases.

Previously, the National Court Administration proposed countermeasures against the citation of AI-generated fabricated statutes and precedents through its "Task Force on Countering the Submission of AI-Generated False Claims and Evidence," which operated from November last year to March this year. The task force concluded that if a litigant or legal representative causes unnecessary litigation costs or delays trials by citing AI-generated false statutes or precedents, they may be ordered to bear all or part of the resulting litigation costs. It also put forward measures allowing trial benches to request disciplinary action from the Korean Bar Association against attorneys who submit AI-generated precedents without verification.

Regarding this, a judge at the Seoul Central District Court remarked, "It is positive that AI lowers the barrier of access to legal services for the general public and assists parties in organizing their arguments." The judge added, "However, if AI incorrectly summarizes judgments or distorts legal principles, citizens reviewing actual rulings might conclude that 'the judge ruled incorrectly,' which could erode public trust in the judiciary." The judge noted, "As the adoption of AI is an irreversible trend, we must contemplate ways to leverage its advantages while filtering out errors."

                                                                                                          Cheon Hyun-bin