When using Artificial Intelligence goes wrong: Judge slams lawyers for legal bungle
AI is definitely not to be trusted to be a legal eagle, a judge found after he was confronted by two non-existent citations in a matter before him
Image: File
Faced with non-existent legal citations in a matter before the Gauteng High Court, Johannesburg, an acting judge asked counsel responsible for it whether the citations constituted artificial intelligence (AI) "hallucinations," to which the red-faced advocate confirmed that “it appears to be so.”
Acting Judge DJ Smit made this discovery when he was about to write his judgment in a matter of Northbound Processing, which wanted to compel the South African Diamond and Precious Metals Regulator to release a refining licence to it.
Judge Smit remarked that while drafting this judgment, it came to his attention that two cases cited in Northbound’s heads of argument do not exist. He invited Northbound’s counsel to clarify the position.
Time pressure was given as an excuse, as the matter came before court as an urgent application. The lawyer explained that he used an online subscription tool called “Legal Genius,” which claimed that it was "exclusively trained on South African legal judgments and legislation".
The court was, however, told that while some non-existent citations (of previous applicable case law) did appear on the heads of argument, the senior advocate who argued the matter did not rely on the non-existent cases during oral argument.
The lawyer who explained the situation to the court accepted full responsibility for the mistakes but emphasised that there was no intent to mislead the court.
The senior advocate (who was not responsible for the non-existent citations) apologised profusely on behalf of Northbound’s legal team. He also explained that he relied upon an experienced legal team (which included two competent junior counsel) upon whom he believed he could (and indeed did) rely.
The senior counsel said he only did a “sense-check” on Northbound’s heads before they were filed and did not have sufficient opportunity to check the accuracy of the citations.
In his judgment on this topic, Judge Smit referred to a recent King’s Bench Division judgment in which the English judge warned against the risks of using AI, especially in legal research, as it could be entirely incorrect and might cite sources which do not exist.
The judge said there are serious implications for the administration of justice and public confidence in the justice system if artificial intelligence is misused.
Judge Smit acknowledged the apologies of counsel in this matter, but he said even negligence in this context may have grave repercussions. He referred the conduct of the legal practitioners to the Legal Practice Council for investigation.
zelda.venter@inl.co.za