Supreme Court Flags AI-Generated Fake Case Laws and Hallucinated Quotes in Petitions, Warns Lawyers on Unchecked Use of Generative AI
The Supreme Court of India has raised serious concerns about a growing problem in the country’s top courts. Lawyers are using artificial intelligence (AI) tools to draft petitions, and this has led to fake case laws and made-up judicial quotes being filed in court.
A three-judge bench of the Supreme Court, led by Chief Justice Surya Kant along with Justice BV Nagarathna and Justice Joymalya Bagchi, issued a strong warning to lawyers. The bench said that careless use of AI could harm the integrity of the judicial process.
“We have been alarmingly told that some lawyers have started using AI for drafting,” Justice Kant said during the hearing.
Fake Cases and Fabricated Quotes Found in Petitions
The bench pointed out several cases where AI “hallucinations” appeared in official court filings. Justice Nagarathna shared one example where a lawyer cited a case that does not exist.
“There was a case of Mercy vs Mankind which does not exist,” she said.
The problem is not limited to completely fake cases. In some petitions, lawyers cited real Supreme Court judgments. However, the quoted portions were not part of the actual rulings. Justice Nagarathna noted that AI had generated false excerpts even when the case itself was genuine.
Justice Kant also mentioned a recent matter before Justice Dipankar Datta. In that case, every legal precedent cited by the lawyer was later found to be fake.
Understanding AI “Hallucinations” in Legal Drafting
AI hallucinations happen when generative AI tools confidently produce false or made-up information. These systems are designed to provide answers, but they do not always check whether the information is accurate. As a result, they can invent cases, quotes, and legal references that sound real but are not.
While AI can make legal research and drafting faster and more efficient, it also carries serious risks. In the legal field, accuracy and credibility are critical. Submitting false information—whether intentional or not—can damage trust in the judicial system and raise ethical concerns for legal professionals.
Also Read : Activision Confirms Call of Duty Warzone Mobile Will Shut Down on April 17
