from Hacker News

Human judgment must remain central to health insurance claims: California law

by bentocorp on 1/6/25, 4:27 AM with 37 comments

• by fragmede on 1/6/25, 5:27 AM

  The headline is misleading. The bill allows AI and algorithms to be used, as long as it doesn't supplant a licensed medical professional deciding (K.1.D), or violate civil rights along with a few other things, but it's not outright prohibited as the headline could be interpreted.

  Section K.1 of SB 1120

  https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...

  (old title was some thing like New California law prohibits using AI as basis to deny health insurance claims)

• by siliconc0w on 1/6/25, 5:23 AM

  There needs to be a consequence for improper denials, regardless of mechanism. Banning AI just means they have to hire people to rubber-stamp.

• by pc2g4d on 1/6/25, 5:36 AM

  None of this would matter if there were real competition in the insurance market, instead of people having to change jobs to change insurance, and not getting a direct say even then.

  As it is, this is a dumb law, and prejudiced against decisions made in silico rather than in vivo.

• by doctor_radium on 1/6/25, 5:30 AM

  Hopefully the law is written broadly enough to include algorithm-driven decisions that existed long before the current "AI" trend.

• by Jimmc414 on 1/6/25, 5:36 AM

  >"For purposes of this subdivision, 'artificial intelligence' means an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments."

  This definition is overly broad and potentially problematic for multiple reasons:

  The definition could encompass simple rule-based systems or basic statistical models. Even basic automated decision trees could potentially fall under this definition. There's no clear distinction between AI and traditional software algorithms. The bill groups "artificial intelligence, algorithm, or other software tool" together in its requirements. This makes it unclear whether different rules apply to different types of automation. Basic automation tools might unexpectedly fall under the AI regulations. The definition focuses on "autonomy" and "inference" without defining these terms. It doesn't distinguish between machine learning, deep learning, or simpler automated systems. The phrase "varies in its level of autonomy" is particularly vague and could apply to almost any software.

  This is legislation that may sound effective and mean well, but the unintended consequences of increased costs and delayed decisions based upon a naive definition of AI seems inevitable.

• by id00 on 1/6/25, 5:09 AM

  I'm really surprised that it's not set on a federal level yet.

  I was working with background checks in the US and it was a rule for a while that every rejection has to go through a real person

• by from-nibly on 1/6/25, 6:23 AM

  Don't tell poeple how to do stuff. Thell them what outcomes they are responsible for. They will figure it out from there.

  If they reject a claim that was legal that should open them up to liability for the results of rejecting that claim.

• by exabrial on 1/6/25, 5:33 AM

  how about we just ban "ai" to decision anything... kinda dumb people put so much faith in something that spits out wrong answers nearly every time.

• by avidiax on 1/6/25, 4:51 AM

  I feel this might be counterproductive.

  An AI and its training data are likely discoverable. Hallway conversations and group meetings are not.

• by aitchnyu on 1/6/25, 5:29 AM

  How is AI defined?

• by mrayycombi on 1/6/25, 7:11 AM

  Even if this headline were true, they'd find another rationale to deny claims if that's their strategy.

  If AI didn't let them deny claims, they'd avoid it. See?

• by tengbretson on 1/6/25, 5:29 AM

  TBH an ai model that is compelled to provide its chain of thought to the patient might be an upgrade over the status quo.