The Supreme Court on Thursday set aside a National Company Law Tribunal (NCLT) order in the insolvency case involving Essel Infraprojects Ltd after finding that the tribunal had relied on "non-existent, fake, and hallucinated" judicial precedents generated by artificial intelligence (AI) tools. The top court also declared that courts must adopt "zero tolerance" towards citing or relying on AI-generated precedents without verification.
A bench of Justices PS Narasimha and Alok Aradhe said, "The production of fake, non-existent, and hallucinated material and its utilisation as precedents in law, is like the release of methyl Isocyanate in the province of law and justice: invisible, insidious, and catastrophic by the time anyone notices. It not only contaminates but takes away the very lifeblood of judicial determination."
Setting aside the NCLT order, the bench said, "It is necessary for Courts to adopt a zero-tolerance mode for producing, citing or using AI-generated precedents without verification. It is a misconduct on the part of an advocate to cite such judgments without verification."
The court added that it was equally a "serious lapse" if a judge relied on fake or hallucinated AI-generated material as legal precedents while deciding a case.
"We have no hesitation in declaring that such a decision is no decision in the eyes of the law, irrespective of whether such material had a direct or indirect bearing on the decision-making. Such decisions are to be set aside even if an iota of fake or hallucinated material enters the decision-making process, as it would violate the sanctity of adjudication," the bench said.
Stressing the need to preserve the integrity of judicial decision-making, the court said, "We reiterate and declare zero tolerance for the Bar as well as the Bench to cite, refer to, or rely on such material. It is also clarified that our judgment shall have no bearing on the rightful use of AI, but on the presentation or reliance on fake or hallucinated material as if it were a court precedent."
The bench said merely declaring a prohibition was not enough and that accountability must follow.
"So far as the responsibility of the Bar is concerned, we direct the Bar Council of India (BCI), being the apex statutory body, to constitute a committee and deliberate on this issue of members of the Bar submitting such fake and hallucinated material before the Court as if they are precedents of law," it said.
The court directed the BCI to treat the issue with the "utmost seriousness", formulate guiding principles to prevent such occurrences and prescribe disciplinary action for violations.
The case arose from an insolvency dispute involving Pooja Ramesh Singh, Jammu and Kashmir Bank Ltd and Essel Infraprojects Ltd. The appellant had challenged an NCLT Mumbai order admitting an insolvency application under Section 7 of the Insolvency and Bankruptcy Code.
Examining the record, the Supreme Court found that several "precedents" cited by the NCLT to support its decision simply did not exist. These included fabricated case names and paragraphs wrongly attributed to genuine judgments. Among them were ICICI Bank Ltd vs Urban Infrastructure Real Estate Ltd (2019) 16 SCC 528 and Sarbjit Singh vs Union Bank of India (2022) 7 SCC 464, both of which the court found to be entirely fictitious citations.
While Jammu and Kashmir Bank filed an affidavit stating that its counsel had not cited those cases and that the NCLT had obtained them through its "own research", the apex court held that the source of the error did not lessen the damage caused to the rule of law.
"More than the inevitable consequence of setting aside such judgments, what is significant for our decision-making is our resolve to adopt AI technology in aid of adjudication, while at the same time asserting and declaring total and absolute control over adjudication, with a human in the loop at every stage," the bench observed.