A new California law (SB 1120) requires a health care service plan or disability insurer, including a specialized health care service plan or specialized health insurer, that uses an artificial intelligence, algorithm, or other software tool for the purpose of utilization review or utilization management functions, or that contracts with or otherwise works through an entity that uses that type of tool, to ensure compliance with specified requirements, including that the artificial intelligence, algorithm, or other software tool bases its determination on specified information and is fairly and equitably applied, as specified.
The new law prohibits the AI, algorithm, or other software tool from denying, delaying, or modifying health care services based in whole or in part on medical necessity. Instead the law requires a medical necessity determination to be made only by a licensed physician or other licensed health care professional competent to evaluate the specific clinical issues involved in the health care services requested by the provider, as provided in existing law, by reviewing and considering the requesting provider’s recommendation and based on the patient’s medical history or other clinical history, as applicable, and individual clinical circumstances.
According to the author of this new law, “recent reports of automated decision tools inaccurately denying provider requests to deliver care is worrisome. While AI has the potential to improve healthcare delivery, it must be supervised by trained medical professionals who understand the complexities of each patient’s situation. Wrongful denial of insurance claims based on AI algorithms can lead to serious health consequences, and even death. This bill strikes a common sense balance that puts safeguards in place for automated decision tools without discouraging companies from using this new technology.”
The new law was supported by several medical organizations including the California Medical Association, California Academy of Family Physicians, California Chapter of American College of Cardiology California Dental Association, California Hospital Association, California Orthopedic Association California Podiatric Medical Association, California Rheumatology Alliance, and others.
There was no opposition to this new law voiced by any organization according to the Bills history.
The California Medical Association, sponsor of this bill, writes while AI tools can improve access to care and assist providers, they have also faced criticism for inaccuracies and biases. This bill addresses those issues by guaranteeing that a provider has final approval of utilization review decisions when AI is being used.
According to a report about this new law by Government Technology Today newsletter, last year, about a quarter of all health insurance claims were denied in California — a reality mirrored nationwide that has stoked public anger toward health care companies, and led to accusations that such decisions lack human empathy.
According to 2024 data from the California Nurses Association, approximately 26% of insurance claims are denied, one of many factors that inspired the law’s primary author, state Sen. Josh Becker, a Menlo Park Democrat.
“In 2021 alone, (nationwide) data showed that health insurance companies denied more than 49 million claims,” said Becker, citing data from the Kaiser Family Foundation. “Yet customers appealed less than 0.2% of them.”
In November 2023, a lawsuit against UnitedHealthcare spotlighted concerns about the misuse of AI in health insurance decision-making, accusing the company of using artificial intelligence to deny claims.
While SB 1120 does not entirely prohibit the use of AI technology, it mandates that human judgment remains central to coverage decisions. Under the new law, AI tools cannot be used to deny, delay or alter health care services deemed medically necessary by doctors.