New Delhi: The AI misuse in courts debate has reached the Supreme Court of India after serious concerns emerged over lawyers and courts relying on fake judgments generated through Artificial Intelligence tools.
The Supreme Court has now informally urged the Bar Council of India (BCI) to create an expert committee to examine the growing problem of AI-generated “hallucinated” case laws being used in judicial proceedings.
The observations came from a Bench of Justice P. S. Narasimha and Justice Alok Aradhe during hearings linked to cases where non-existent judgments were reportedly cited in court records. The Court stressed that judicial integrity cannot be compromised because of unchecked AI usage.
Supreme Court Raises Alarm Over AI Misuse in Indian Courts
The Supreme Court said the increasing use of AI tools in legal drafting and research is becoming a serious concern, especially when such tools generate fake or incorrect case laws.
According to reports from court proceedings, the Bench orally asked the Bar Council of India to set up a committee of experts, including technical specialists, to study the issue and submit recommendations.
The Court reportedly observed:
“We want the Bar Council of India to constitute a committee of experts…”
However, the Bench clarified that no formal judicial order has been passed yet.
AI Misuse in Indian Courts: Why the Supreme Court Is Concerned
The concern started after courts noticed that some submissions contained judgments that either never existed or were wrongly cited because of AI-generated legal research.
The Supreme Court warned that such practices directly affect the credibility of the justice delivery system. Judges also highlighted that simply apologizing later cannot remove accountability in sensitive legal matters.
The Bench further noted that if judges themselves unknowingly rely on fabricated AI-generated material, the consequences could become even more serious.
Court Says Fake AI Citations May Amount to Misconduct
In earlier hearings connected to the same issue, the Supreme Court stated that relying on fake AI-generated judgments is not merely a technical mistake.
The Court observed that decisions based on non-existent judgments could amount to “misconduct” and may attract legal consequences.
The remarks came while examining an Andhra Pradesh matter where a trial court reportedly relied on fake precedents generated through AI systems. The High Court later identified those citations as fabricated.
Growing Global Concern Around AI in Legal Systems
India is not alone in facing this challenge.
Globally, courts have increasingly flagged problems caused by unchecked AI usage in legal work. In the United States, several lawyers and prosecutors have already faced penalties for submitting AI-generated fake citations in court proceedings.
Legal experts say generative AI tools can sometimes “hallucinate” and create fictional judgments, laws, or legal references that appear real but do not actually exist.
Because of this, courts worldwide are now emphasizing human verification before using AI-assisted legal research.
Supreme Court’s Centre for Research Already Prepared White Paper
During the hearing, senior advocate and amicus curiae Shyam Divan informed the Bench that the Supreme Court’s Centre for Research and Planning has already prepared a white paper on AI usage in the judiciary.
The white paper reportedly contains recommendations and possible safeguards for responsible AI use in court processes. The matter is expected to be heard again later this month.
Need for Responsible AI Use in Indian Judiciary
Experts believe AI can still help the judiciary in research, document management, and reducing workload if proper safeguards are introduced.
However, the Supreme Court’s latest observations make it clear that AI cannot replace professional legal verification.
The Court’s message is simple: technology may assist legal professionals, but responsibility and accountability must always remain with humans.
Read also: Turning Point for Indian Lawyers: Supreme Court Questions BCI’s Ban in Bar Council Elections
