from Hacker News

Excessive load on NTP servers

by BCM43 on 12/20/16, 2:01 PM with 150 comments

  • by easytiger on 12/20/16, 2:21 PM

    Wait.. they are saying the app itself is making NTP requests?

    > Confirmed - starting up the iOS Snapchat app does a lookup to the domains you listed, and then sends NTP to every unique IP. Around 35-60 different IPs.

    Hmm. Is that a fraud prevention thing or something? No way on earth a user app should be getting its own time

  • by sschueller on 12/20/16, 2:54 PM

    Why on earth would you do that?

    If you want to prevent users from altering their time use your server and do a time compare with your server.

    NTP can be easily intercepted and altered so it would make a lot more sense to do this via a encrypted certificate pinned communication path increasing my work load drastically to alter the time.

    I snapchat going to pay for the DDOS they created?

  • by Declanomous on 12/20/16, 2:38 PM

    For whatever reason, ntppool.org is blocked at my work.

    And of course, you don't get the page that states why when the website is served via https. Not that I need to see the page to know it was either blocked for "hacking" or "entertainment", and I'm guessing it's not entertainment.

    Edit: This probably explains why our clocks have been off by 45 minutes since Monday. I guess it will be entertaining to see how long it takes for IT to figure this one out.

  • by acqq on 12/20/16, 2:33 PM

    According to the forum, the pattern matched this third-party library:

    https://github.com/jbenet/ios-ntp

    Specifically, all the servers(!) from here are contacted: https://github.com/jbenet/ios-ntp/blob/master/ios-ntp-lib/Ne...

    Note that the library author wrote:

    "ios-ntp is often (mostly?) used to make sure someone hasn't fiddled with the system clock. The complications involved in using multiple servers and averaging time offsets is overkill for this purpose. The following skeleton code is all that is needed to check the time."

    And that "skeleton" contacts just "time.apple.com"

    But the library really has the default possibility of contacting a lot of the ntp.org servers from a big list ("createAssociations" with no parameters!) and it's bad.

    As we know, the developers like to just "copy-paste" whatever is where. Or use any defaults. "Hey it works."

  • by coleca on 12/20/16, 2:57 PM

    FWIW my teenage daughter has been complaining about this latest Snapchat update for iOS the past couple days. It constantly crashes and causes the phone to reboot itself. Looking at Twitter, there's tons and tons of people reporting the same issue, so it seems pretty widespread. Wonder if it's related to this NTP issue.

  • by sateesh on 12/20/16, 7:33 PM

    It is interesting to read through the whole thread in a chronological order starting from the first message: http://mailman.nanog.org/pipermail/nanog/2016-December/08952...

    It took 4 days, to zero on the root cause. As is usual in a complex scenario like this there are a few false positives, some suspects abusing the protocol and alas final redemption. Amazing work by a dedicated group of technical folks in coordinating (just via emails, I suppose) and tracing the root cause.

  • by lima on 12/20/16, 2:36 PM

    Worst part is that they did not bother to use a vendor zone.

  • by mark-r on 12/20/16, 4:46 PM

    This happens often enough that Wikipedia has a page devoted to it: https://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse

    The first one I had heard of was Netgear vs. UW-Madison.

  • by gbrown_ on 12/20/16, 2:26 PM

    For all of Apple's App Store vetting one would think this kind of behavior would have thrown up a flag at some point no?

  • by _RPM on 12/20/16, 7:06 PM

    And to think that SC's engineering is praised among college kids is laughable.

  • by Faaak on 12/20/16, 10:51 PM

    I wondered why I was seeing so much packet loss on my IP: http://mrtg.vi-di.fr/krootservers.ping.html

    Guess I know why now..

  • by thejosh on 12/20/16, 3:55 PM

    Yeah, it's been really hit and miss here in AU for a few people I know.

  • by sstevo66 on 12/22/16, 12:51 AM

    I do some work for the Network Time Foundation and we were not contacted by snapchat as far as I know. Anyone have a contact there, they probably need our help.

  • by 1_2__3 on 12/20/16, 3:03 PM

    I for one am shocked - shocked! - that Snapchat would be the kind of company to be cavalier about this kind of thing.

  • by known on 12/20/16, 4:47 PM

    Captcha should fix it