An ongoing case has highlighted the risks of relying too heavily on AI tools like ChatGPT for legal work. A federal magistrate judge has recommended a $15,000 sanction against attorney Rafael Ramirez for submitting court briefs containing fictitious case citations generated by an AI chatbot (The Register, 2025). Ramirez, who filed these briefs in October 2024, later admitted he was unaware that AI tools could fabricate legal references. This situation serves as a stark reminder for all professionals—especially in the legal field—to be extremely cautious when using AI for research and document drafting.
We’ve personally encountered this issue when experimenting with ChatGPT as a web search tool to find legal news. Instead of providing accurate results, it fabricated cases and even invented news sources. AI hallucinations, where the model generates plausible-sounding but entirely false information, are a serious concern. While ChatGPT has improved significantly since late 2024, it still isn’t perfect, particularly when it comes to deep research.
A good practice to mitigate these risks is to always ask AI tools to link directly to their sources. ChatGPT has become more consistent in citing sources, but that doesn’t mean blind trust is warranted. Always double, triple, and even quadruple-check your work. No matter the tool—be it AI, search engines, or traditional research methods—verification is key.
Ultimately, Ramirez’s penalty was relatively light, but this case raises broader concerns about the reliability of AI in professional settings. While ChatGPT has improved, it still requires oversight. The takeaway? AI can be a useful assistant, but never a substitute for thorough, human-verified research.