from Hacker News

Ask HN: Anyone else having bitflips in Azure Network traffic?

by klaas- on 10/28/23, 12:47 PM with 12 comments

It seems we are having bit flips inside of packets coming into Azure (Germany West Central) via our Express Route. Anyone else seeing this?
  • by dharmab on 10/28/23, 10:05 PM

    I did back in ~2017 or so while I was working at $BIGAZURECUSTOMER. It was tough getting a hold of the right person in support, so we pulled some backchannel strings. Turns out an update to their hardware had disabled ECC on some switches.
  • by klaas- on 10/28/23, 2:29 PM

    So it seems this is their software defined network, the packets still look good when entering via the microsoft edge routers and they get bitflips on their way into our express route
  • by shrubble on 10/28/23, 6:52 PM

    Can you do Wireshark style packet captures somewhere that show it? Very likely that would be what support would ask for.
  • by klaas- on 10/30/23, 12:13 PM

    I also have a follow up to this topic. I would say it looks like in Azure nobody is verifying TCP checksums. A bitflip in tcp payload should have been noticed at the NIC/tcp level but it reaches my application with corrupted data.
  • by johnwheeler on 10/28/23, 9:01 PM

    Like from neutrinos?