Alexander Bein, Andrew Gann, Armina Manning and Sandra Smith Thayer | McGuireWoods
Policyholders increasingly turn to artificial intelligence (“AI”) platforms to answer insurance coverage questions. This is risky because: (1) AI models hallucinate legal content at alarming rates; and (2) AI cannot apply nuanced, fact-specific and jurisdiction-specific legal analysis. As OpenAI Foundation recently acknowledged, “ChatGPT . . . neither has nor uses any degree of legal knowledge or skill.” See
https://fingfx.thomsonreuters.com/gfx/legaldocs/xmvjydomqpr/Nippon%20Life%20v%20OpenAI%20motion%20to%20dismiss.pdf. Any policyholder who receives a “no coverage” answer from AI should consult experienced coverage counsel before accepting that conclusion.
The scenario is increasingly common. A business suffers a loss (a cyberattack, a construction defect claim, a product liability suit) and wonders: Does our insurance cover this? Rather than call a lawyer, they open ChatGPT or some other AI platform. If it says “no coverage,” the natural instinct is to move on. But that instinct is often misguided. Policyholders feeding coverage questions into AI models and receiving plausible-sounding negative answers are walking away from claims worth pursuing. And the reasons for these false negatives are structural and predictable.
AI Platforms Are Conservative and Are Not Designed to Confidently Predict Policyholder Success
Large language models are trained on massive datasets scraped from the internet, including from publicly available legal materials, treatises, court opinions, and industry publications. Those sources often emphasize general rules and may fail to identify exceptions applicable to a given policyholder’s specific coverage situation. As a result, AI often misses nuanced, pro-coverage arguments that policyholder counsel may identify based on experience, skill, and the exercise of judgment not available to AI.
Additionally, AI tends to avoid overstatement: in the case of uncertainty, AI is designed to hedge rather than confidently predict a policyholder victory. AI is more likely to say “coverage may be excluded” than “coverage clearly exists” when the law is unsettled or policy language is ambiguous. Insurance law is also highly fact-specific and may vary significantly from jurisdiction to jurisdiction, so a small change in facts or policy wording can completely change the answer. Without the necessary context provided by complete policy documentation, applicable law, and the underlying facts and circumstances, AI often defaults to a cautious analysis that does not project confidence in a successful outcome.
AI Platforms Often Get the Law Wrong
When it comes to accuracy on complex legal questions, the track record of AI platforms is sobering. A Stanford study found that general-purpose AI models (i.e., publicly available models not specifically developed for legal enterprise settings) hallucinate (i.e., fabricate legal content) at rates of 58% to 82% on legal queries; specialized legal AI tools hallucinated between 17% and 33% of the time. See Magesh et al., Hallucination-Free?, Stanford HAI (2025).
The models routinely apply the wrong state’s law, cite nonexistent cases, and attribute holdings to judges who never wrote them, all with the same confident tone. AI does not know when it is wrong. For example, in Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023), an attorney submitted a brief containing ChatGPT fabricated cases. When challenged by the court, the attorney asked ChatGPT to confirm the cases were real; it assured him they were.
One AI hallucination case-tracking website has estimated that well over 1,600 AI-generated fabrications have been submitted to courts. See https://www.damiencharlotin.com/hallucinations/. If even seasoned attorneys are failing to catch these errors, consider a business owner or risk manager with no legal training who treats an AI coverage opinion as the final word.
The Cost of a False Answer
The risk here is that when a policyholder relies solely on a “no coverage” answer from an AI platform, without consulting a coverage attorney, they may not file a claim or push back on an erroneous denial of coverage from their insurer. The claim dies quietly and insurance money is left on the table.
A proper coverage analysis requires understanding the specific policy language (including all endorsements), applicable law, facts of the claim, and policy interpretation principles (e.g., ambiguities are construed in favor of coverage, exclusions are interpreted narrowly against insurers, and insurers have the burden to prove exclusions apply). AI lacks the legal knowledge, skill, and judgment to reliably apply these bedrock principles.
What a Coverage Attorney Sees That AI Does Not
The American Bar Association has recognized that lawyers must understand technology risks. ABA Formal Opinion 512 (July 2024) requires competence, confidentiality, and independent verification when using generative AI. If AI is not reliable enough for trained attorneys without careful oversight, it is certainly not reliable enough for policyholders to use as a substitute for experienced coverage counsel.
Experienced policyholder-side attorneys bring something AI cannot replicate: deep legal knowledge, skill, and experience in interpreting complex insurance policy provisions, an understanding of jurisdiction-specific coverage laws and issues, and the best way to marshal the facts and craft arguments based on the policy language and the law in order to maximize coverage under your insurance policies.
The Last Word Should Not Belong to a Machine
AI can summarize documents and draft correspondence in seconds. But reliable coverage opinions require judgment, skill, and experience—something AI admittedly does not possess.
Key Takeaway: Any policyholder who has received a denial of coverage from their insurer, or a “no coverage” answer from AI, should not accept that conclusion without question. Experienced coverage counsel could be the difference between absorbing a loss and recovering potentially millions in insurance benefits.
An algorithm is a tool, not a lawyer. It should not have the last word on an insurance claim.
When one of your cases is in need of a construction expert, estimates, insurance appraisal or umpire services in defect or insurance disputes – please call Advise & Consult, Inc. at 801.641.8304, or email experts@adviseandconsult.net.