from Hacker News

Seaborn bug responsible for finding of declining disruptiveness in science

by ossicones on 2/25/24, 2:17 PM with 72 comments

• by ironSkillet on 2/25/24, 4:26 PM

  I saw this headline and my first thought was that someone was claiming that a mind impacting virus that evolved in the ocean was causing scientists to do research with less ambition. Which is of course ridiculous lol. But a bug in a visualization library impacting science is also ridiculous.

• by raziel2701 on 2/25/24, 5:52 PM

  Science communication must be at an all-time low. I initially thought the paper was about a sea-borne pathogen being responsible for a decline in disruptiveness in science, which is a crazy statement.

  Then I thought that it was a paper claiming that a bug in the seaborn plotting library in python was responsible for the decline in disruptiveness in science, which is absurd!

  Finally I understood, that this is a paper that is debunking another meta paper that claimed that disruptiveness in science had declined. And this new, arxiv paper is showing that a bug in the seaborn plotting library is responsible for the mistake in the analysis that led to that widely publicized conclusion about declining disruptiveness in science. oh boy so many levels...

• by bumbledraven on 2/25/24, 3:49 PM

  The seaborn issue linked in the paper, “Treat binwidth as approximate to avoid dropping outermost datapoints” (https://github.com/mwaskom/seaborn/pull/3489), summarizes the problem as follows:

  > floating point errors could cause the largest datapoint(s) to be silently dropped

  However, the paper does not contain the string “float”, instead saying only:

  > A bug in the seaborn 0.11.2 plotting software [3], used by Park et al. [1], silently drops the largest data points in the histograms.

  So at the very least, the paper is silent on a key aspect of the bug.

• by daveguy on 2/25/24, 3:54 PM

  Seaborn is a visualization library. No statistical tests should have been done with seaborn as an intermediate processing step. I guess they used some of the convenience functions as part of the data analysis. Seaborn is a final step tool, not a data analysis tool. That's an embarrassing lesson to learn post-publication.

• by light_hue_1 on 2/25/24, 4:28 PM

  I hope that all the publications that celebrated the original work, like the Economist https://www.economist.com/science-and-technology/2023/01/04/..., Nature's news service https://www.nature.com/articles/d41586-022-04577-5, the FT https://www.ft.com/content/c8bfd3da-bf9d-4f9b-ab98-e9677f109..., and others spend as much time on correcting the record as they did on promoting the idea that science is broken.

  And I hope the original authors tell Nature to retract their paper. It's already highly influential unfortunately.

• by sitkack on 2/25/24, 3:51 PM

  This image is the best illustration of the flaw https://arxiv.org/html/2402.14583v1/x1.png

  On mobile and can’t read the rest of the paper, the impact could be massive.

• by moh_maya on 2/25/24, 10:22 PM

  The submission was flagged, and I am not sure I understand why since the only (negatively) critical discussion I see is on the ambiguity over the title in the HN submission; flagging a submission appears to take it off the HN homepage, and I feel a title ambiguity in the face of the significance of the submission itself isn’t a strong reason for removing the submission from HN? :)

  There are (at the time of posting this comment) no comments raising any substantive issue with the arxiv submission itself (which ofc has to go through the peer review process of publication, and hopefully the original authors will respond / rebut this new article) - so curious why its been flagged? It’s not dead, so cannot vouch for it.

  If folks in the HN community who have flagged it have done so because there are serious issues with what the paper is asserting, please comment / critique instead of just flagging it. If it’s because of the ambiguity in the title, I hope @dang and the moderators editorialize - there are some valuable comments in this thread that helped me understand what the issue is and what the bug is!

• by math_dandy on 2/25/24, 5:21 PM

  Damn hipsters should just use matplotlib like the rest of us.

• by keenmaster on 2/25/24, 3:49 PM

  Bizarre. How do people make such big, splashy findings that can mess with people’s sense of optimism about science and innovation, without doing the simplest types of checks on their data and methodology.

• by dkasper on 2/25/24, 3:55 PM

  Sounds like a plot point from three body problem.

• by bmitc on 2/25/24, 6:58 PM

  I wonder how much bad science has occurred due to the acceptance of Python as the lingua franca.

• by KRAKRISMOTT on 2/25/24, 3:49 PM

  The graphing library caused this?

• by sergers on 2/25/24, 3:55 PM

  Reading the comments here hilarious.

  Like others, expecting a wildy different article...

• by rhelz on 2/25/24, 5:20 PM

  Of course, it has nothing to do with rampant fraud, unreproducible results, incentive structures which reward the number of papers over the quality of papers, having researchers spend their prime scientific years writing grant proposals instead of actual research...

  ...nor does it have anything to do with tech companies hoarding cash by the trillions of dollars oversees instead of spending it on R&D, and even what R&D they internally produce they have no incentive to publish or productize, because virtually no new business will be more profitable than the monopoly business they already have...

• by asplake on 2/25/24, 3:26 PM

  Seaborn??? Typo surely

  Edit: Not mentioned in the abstract but it is in the main paper. Editorialised title.