from Hacker News

Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

by 4512124672456 on 9/22/21, 1:39 PM with 460 comments

by gwerbret on 9/22/21, 3:07 PM
I'm going to make what is probably a controversial statement, which is that, in my experience, most meta-analyses are trash.
A meta-analysis is usually grounded on the assumption that an aggregation of multiple, well-conducted studies can provide significant information that may not be evident by taking each study individually. Essential to this assumption is the premise that those studies included in the meta-analysis be scientifically rigorous in themselves. Unfortunately, this is almost never the case. The great utility of the meta-analysis in the clinical literature is such that there have emerged a series of standards for how to conduct such analyses, such as controlling for bias, controlling for variability, etc. These seldom include actual assessment of the scientific quality of the studies, as this is harder to standardize. Consequently, there is a preponderance of meta-analyzes that come to conclusions that are simply not justified, because the studies themselves were not scientifically rigorous.
I believe this applies to the situation of ivermectin, and it just so happens that this is the thrust of the point raised in the Nature article cited here.
by ashtonkem on 9/22/21, 2:55 PM
The other takeaway is that preprint publication is a pathway ripe for abuse. The major paper that drove most of this situation, and the meta analysis, appears to have been a straight up fraud. But it got circulated a lot as a preprint to people who didn’t understand the difference between a preprint and a peer reviewed paper. If the preprint had been more private, this might have been averted.
by nradov on 9/22/21, 2:12 PM
Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article.
https://journals.lww.com/americantherapeutics/fulltext/2021/...
Ultimately we need a real large scale controlled trial to settle the issue so I'm looking forward to seeing results from NIH ACTIV-6.
https://www.nih.gov/research-training/medical-research-initi...
by cyounkins on 9/22/21, 8:49 PM
There is a lot of confusion on whether there actually was a significant increase in calls to poison control centers due to ingestion of ivermectin.
This [1] official Mississippi government document says, "At least 70% of the recent calls have been related to ingestion of livestock or animal formulations of ivermectin purchased at livestock supply centers."
But then the AP [2] seemed to say that was incorrect: "The Associated Press erroneously reported based on information provided by the Mississippi Department of Health that 70% of recent calls to the Mississippi Poison Control Center were from people who had ingested ivermectin to try to treat COVID-19. State Epidemiologist Dr. Paul Byers said Wednesday the number of calls to poison control about ivermectin was about 2%. He said of the calls that were about ivermectin, 70% were by people who had ingested the veterinary version of the medicine."
Does anyone have additional clarification?
[1] https://msdh.ms.gov/msdhsite/_static/resources/15400.pdf
[2] https://www.sfgate.com/news/amp/Health-Dept-Stop-taking-live...
by motoboi on 9/22/21, 2:59 PM
Someone said it and I totally agree:
Society is not ready to watch science in realtime.
by mgamache on 9/22/21, 9:32 PM
And yet the Authors of this letter fail to address the most cited meta-analyses (by Dr Lawrie). She's publicly stated the removing the Elgazzar data did effect her numbers, but not the conclusion. Also, if you read the paper she downgrades the Niaee data for several potential biases.
https://pubmed.ncbi.nlm.nih.gov/34145166/
by buescher on 9/22/21, 3:03 PM
The lesson of ivermectin: a lot of medical studies that go uncriticized because they're less controversial probably have problems similar to the problems of the retracted Elgazzar study and the other studies mentioned. Why else would these people think they would get away with it?
by mechE321 on 9/22/21, 3:26 PM
The Bayesian gambler in me wants to think this:
The studies on ivermectin seem to be split between "good effect" and "no effect," and there don't seem to be any (by my extremely informal review! going off of memory here) in the camp of "bad effect."
Seems reasonable to take ivermectin as a decent gamble to me while we wait on the dang science to get its head out of its butt.
by defaultprimate on 9/22/21, 5:29 PM
Studies are irrelevant at this point, when we have population level data of over a billion people across India and Africa, and the massive disparities between states/countries that utilized Ivermectin, and those that did not. The drug is safe enough to be used based on this data alone.
https://www.thedesertreview.com/opinion/columnists/indias-iv...
https://www.thedesertreview.com/news/national/indias-ivermec...
https://www.thedesertreview.com/opinion/columnists/indias-iv...
https://www.thedesertreview.com/opinion/columnists/indias-iv...
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7968425/
https://journals.lww.com/americantherapeutics/fulltext/2021/...
by programmarchy on 9/22/21, 2:46 PM
How does the current field of studies compare to Remdesivir? Remdesivir is a repurposed antiviral (originally designed for Ebola) with fairly low efficacy, and very severe side effects, and it breezed through FDA emergency authorization. I don't understand why it seems that the bar is set so low for Remdesivir, but so high for Ivermectin.
by ndr on 9/22/21, 2:34 PM
Mind that this is correspondence, not a peer reviewed paper.
For what is worth most meta-study will check if they reach the same results leaving-n out (typically one), but I agree that they could do much better accessing the underlying data itself.
by pier25 on 9/23/21, 4:43 AM
Here in Mexico most doctors are giving Ivermectin to their COVID patients.
by hackingforfun on 9/22/21, 3:46 PM
With all the criticism of the mRNA and other COVID vaccines, which actually do have science backing them up, what is the science that would make ivermectin actually work? It's an antiparasitic drug. I see a lot of criticism of the COVID vaccines, especially from the right, but then they are fine taking hydroxychloroquine or ivermectin if there is any glimmer of hope that they will work. I just don't get it. Maybe, just maybe, it's not all some big conspiracy, and the vaccines were actually designed to help people? Is that so far fetched? Doesn't taking something random like hydroxychloroquine or ivermectin seem more far fetched? It does to me.
by X3h7P4icntav on 9/23/21, 11:32 AM
The question of whether ivermectin treats Covid19 in humans is utterly independent of vaccines, HCQ, the state of modern science, whether ivermectin treats other diseases, ivermectin's use in animals, politics, any corrupt ivermectin studies (if you ignore them), and the price of cheese on the moon.
I have learned there are many idiots among us. Many, many idiots.
by atty on 9/22/21, 8:37 PM
What I find almost more interesting than the article itself is the comments that show a non-zero portion of the HN community (which I think is generally more intellectually curious and educated than average) are actually strongly arguing that ivermectin has strong effect, and even crazier, that it may be safer or more effective than the mRNA vaccines.
Is this from being incapable of following science and drawing rational conclusions? Or is it more of a tribal thing, where they are exposed to a biased subset of information/misinformation, and are now emotionally invested in the success of ivermectin because their tribe is?
Either way, like someone else in the comments said, these past 2 years have shown us that laypeople are incapable of following active science in real time and drawing reasoned conclusions. Frankly, this is probably true of everyone who is not an expert in the field in question. We need organizations like the CDC and FDA to be much better about their messaging (remember the no-mask debacle? Great way to lose credibility, guys and gals), and we need much better tools to shut down the spread and weaponization of misinformation from the anti-vax crowd et al. I honestly don’t have any idea of how either of those get fixed, however.
by lez on 9/22/21, 2:52 PM
Looking at the greyed out comments it seems HN is not pro, but against scientific debate. Questioning science IS part of the scientific method.
https://c19ivermectin.com/ is a very adequate counter-argument to the article, and it's unfair to the whole scientific community that comments are greyed out that mention it.
by sydthrowaway on 9/23/21, 12:40 AM
There is no good faith argument against the vaccine.
Just take it.
by rory on 9/22/21, 2:55 PM
Ivermectin probably doesn't do anything to fight Covid-19, but the American media has really shown their corruption in the way they've treated it.
It's clearly a safe drug to take in human-designed doses, and it's cheap to produce. Laughing at people for poisoning themselves with "horse dewormer" instead of pointing out that they are turning to the vet store because their access to medicine has been marginalized is sick.
And maybe it does help, I don't know. Unproven != disproven.
by eigengrau5150 on 9/22/21, 1:43 PM
I've suspected that for years. I remember my undergrad biology prof coming down on students depending on meta-analysis and insisting they were no substitute for attempting to replicate the experiments themselves.
by input_sh on 9/22/21, 2:48 PM
https://archive.is/9EVIv
by EnlightenedBro on 9/22/21, 3:01 PM
For those interested to learn more about Ivermectin, watch JRE#1671 podcast episode. Lots of interesting information covered about the drug by 2 biologists. It may change your views on this for the better, or the worse.