from Hacker News

Man Allegedly Raped in Jail After AI Wrongly IDs Him as Suspect Despite Alibi

by parkaboy on 1/24/24, 3:35 PM with 91 comments

• by Cheer2171 on 1/24/24, 4:45 PM

  And if you're confused about how a system could even be deployed if the accuracy rate is so bad it leads to a majority of those identified as criminals being false positives... This can result from just measuring accuracy rates, instead of sensitivity and specificity or related error metrics. Say a classifier has a 99.9% accuracy rate for any given photo. If used to identify criminals, and if only 0.1% of the population are criminals, this will lead to a false arrest rate of 50%.

  It is a basic statistical fallacy literally called the prosecutor's fallacy, because it gets misued so much in criminal law, also called the base rate fallacy. 99.9% accuracy sounds high, and is above what state of the art performance is for noisy real-world surveillance data.

  But imagine this system is used on a city with a population of 1 million people. With an accuracy rate of 99.9%, the system will correctly identify a criminal as a criminal 99.9% of the time. But the actual proportion of criminals is, say, 0.1% or 1,000 criminals out of 1 million people.

  When the system scans all 1 million people, it correctly identifies about 999 of the 1,000 criminals (99.9% of 1,000). But here's the catch: it also mistakenly identifies 0.1% of the 999,000 innocent people as criminals. That's 999 false positives. So, the system ends up flagging nearly 1,998 individuals (999 true criminals + 999 false positives) as criminals.

• by saagarjha on 1/24/24, 3:49 PM

  It’s kind of insane that’s we have a justice system that’s so insane that it’s normal to receive “extrajudicial punishment” and that there are people who actively enjoy and secretly, and sometimes not-so-secretly, wish that this mechanism remains in place.

  (We should also probably not rely on AI to put people in that place.)

• by bevacqua on 1/24/24, 3:47 PM

  "A computer can never be held accountable, so has increasingly been used to make management decisions." — IBM slide from 1979.

  AI didn't wrongly convict anyone. People leveraging AI did.

• by bmitc on 1/24/24, 4:06 PM

  > The judge then agreed to dismiss the charges against him.

  > However, just hours before he was released from jail

  How does a person not just walk out of the courtroom after having charges dismissed against them? This case is harrowing.

  Also, there are multiple levels of failures here. Why is a person accused of robbery, who just had charges dismissed against them, in jail in the first place, and furthermore, why are they being housed with violent criminals? Why were they arrested purely on the basis that they look like someone, as determined by an automated system, with no further investigation or oversight?

• by rbetts on 1/24/24, 4:02 PM

  This link includes the lawsuit's documentation and accuses more than negligent use of facial recognition software - including manipulating/pre-briefing an employee presented a photo line-up.

  https://www.houstonpublicmedia.org/articles/technology/2024/...

• by drdeca on 1/24/24, 3:55 PM

  Sometimes I think that every square inch within jail/prison buildings should be constantly under video recording so that if any rape occurs, there can be no question as to whether it occurred, so that the rapist can be executed with no risk of a false conviction.

  That’s probably not actually a sound policy, but, I think it is important to bring the average number of prison rapes per year to basically zero, very quickly, and this is the first thing I think of towards that goal.

  ... I suppose even with plenty of footage there may be the possibility of erring in determining whether it was consensual?

• by enobrev on 1/24/24, 3:48 PM

  So many things went wrong here, on so many levels. Our systems are falling apart and we're not doing enough about it. We definitely shouldn't be relying on companies like this to fill the gaps.

• by jMyles on 1/24/24, 4:02 PM

  One of the things that "innocent until proven guilty" needs to mean is that pre-trial confinement is sufficiently comfortable, safe, and accommodating so as not to be confused for punishment, deterrence, or any other outcome of a judicial proceeding.

  Putting pre-trial detainees in the same population as convicted prisoners (ie, county jails) is very strange.

• by ashleyn on 1/24/24, 3:46 PM

  Horrendous. If more people took seriously the idea they could be incarcerated on clerical errors, perhaps we would see long-overdue and long-needed reforms in our justice system.

• by rhplus on 1/24/24, 3:52 PM

  So “AI” is the problem and not poor police processes and lack of oversight in crowded jails. Got it.

• by ETHisso2017 on 1/24/24, 3:44 PM

  In the future, they won't read you your bill of rights, they'll read you your terms of service

• by highwaylights on 1/24/24, 3:58 PM

  $10 million seems like very little for this man to be asking for under the circumstances.

• by m3kw9 on 1/24/24, 4:04 PM

  What’s crazy is he(60) wasn’t even there for long and was already raped, it must be extreme bad luck or this occurs way more than is reported

• by fennecfoxy on 1/26/24, 11:59 AM

  Interesting that they make a point specifying "allegedly". I think all rape claims should be treated as allegations, of course. That's how justice works, figuring out exactly what happened. But articles with male victims have gone from not even calling it rape to now expressing it was "alleged". Other articles would just headline that someone was raped...

• by hn_throwaway_99 on 1/24/24, 3:46 PM

  The fact that all it takes is one company with a wrong ID to arrest someone is one thing.

  But that being taken to jail in America can result in violent prison rape (and worse, that it is often met with jokes and snickers) shows a very sad, disgusting part of our society.

  The article says he's suing EssilorLuxottica but I also hope he is able to sue the crap out of the government that incarcerated him.

• by gigel82 on 1/24/24, 4:49 PM

  Reminds me of Please Hold (a 2020 short, on HBO Max); here's a trailer: https://www.youtube.com/watch?v=8MTOyyKKCSg

• by NoMoreNicksLeft on 1/24/24, 3:50 PM

  If we just do something about this horrible facial recognition software, only the guilty will be raped in jail awaiting trial.

• by lifestyleguru on 1/24/24, 3:53 PM

  That's exactly the ethical and competence level I would expect from a company like Luxottica.

• by feverzsj on 1/24/24, 4:01 PM

  I thought AI is experimental at best. No serious job should be exclusively done by AI.

• by par on 1/24/24, 3:51 PM

  we're truly living in a dystopian hell.

• by Teknomancer on 1/24/24, 5:04 PM

  I came here to find comments and I had hoped answers as to just how exactly a man walks into a DMV in Texas and gets arrested for a crime that occurred in California?

  Why is nobody asking this question??

  It seems like that should be a substantial part of the story.

• by paxys on 1/24/24, 3:46 PM

  So it would be okay to rape him in jail if he was in fact correctly convicted?