from Hacker News

DNSimple DDOS Attack

by dedene on 12/1/14, 7:47 PM with 114 comments

  • by whafro on 12/1/14, 9:01 PM

    We're kinda tied into DNSimple since we use an ALIAS record for our bare/naked/root domain. Amazon's Route53 supports aliases, but via a 301 redirect, which doesn't work in an SSL context (without browser warnings).

    Nonetheless, we just spun up a Route53 zone, exported our zone from DNSimple, imported to Route53, and hand-migrated our ALIAS records to static A records in the new zone.

    Not perfect or permanent, but we've gotten around the outage. Also, I just learned that pointhq has (seemingly-undocumented) support for ALIAS records in the same style as DNSimple, so this could be another avenue to explore.

  • by jameskilton on 12/1/14, 9:13 PM

    We can watch this happen live @ http://map.ipviking.com/

    Fascinating traffic floods from various locations, but the attack is not continuous.

  • by webandtech on 12/2/14, 3:39 AM

    Free solution that worked for me: Set up a free account on cloudflare.com, duplicate all dns records (thankfully I have a simple setup)... but next time I will keep a backup zone file!

    FYI - Instead of an Alias record on DNSimple, CloudFlare will allow a CNAME record for the root domain using "CNAME flattening".

    You can now set CloudFlare's DNS service to "bypass Cloudfare" on all records by clicking the icon so you don't get any of their magic (unless you want it).

    Then add CloudFlare's 2 nameserves to your domain as your first 2 name servers. No need to remove dnsimple's name servers.

    Now you have 2 DNS providers in case one fails, just make sure the records are the same across them both!

  • by scott_karana on 12/1/14, 10:27 PM

    DNS is so straightforward, so easily distributed, and so fundamental, that I'm always astounded when it's a single point of failure for so many operations.

    I wonder how many of the affected companies do have redundant appservers and load balancers, but missed this piece of the puzzle...

  • by Cantdog on 12/1/14, 11:47 PM

    Can someone help me understand what happens to email sent to a domain hosted by DNSimple while it's down?

    I'm hoping it will get queued by the sending server, and make it's way back when DNSimple is up and running. Is that correct?

  • by zuccs on 12/2/14, 2:52 AM

    I moved from Zerigo to DNSimple, and it's been awesome until now!

    What can you do to prevent this in future? Can you run multiple DNS providers simultaneously? So, ns1/ns2 go to DNSimple, and ns3/ns4 go to another provider?

  • by aberoham on 12/1/14, 10:22 PM

    If you have an active DNSimple web UI session (or API key) you can change your root nameservers by hitting their web tier directly at 50.31.213.210.

    We've successfully switched our domains over to nsone.net.

  • by ericskiff on 12/1/14, 10:10 PM

    For anyone else who needs to mitigate this in a hurry:

    Set up a new account on another host that does ALIAS records (I used pointDNS)

    Create your new record without much in it

    Change your nameservers on your domain now - they'll take time to propagate

    Fill in the records on your domain. If you can't remember them, print out most of your existing records with

    dig yourdomain.com ANY

    Add the rest of the records to pointDNS

    Wait for the new Nameservers to propagate (0-24 hours - it took 15-30 min for us on a small-medium traffic domain today during sales crunch)

  • by englishm on 12/2/14, 5:00 AM

    Here's where you can request your cached SERVFAILs be flushed from Google's public DNS (i.e. 8.8.8.8): https://developers.google.com/speed/public-dns/cache

  • by dedene on 12/1/14, 10:49 PM

    "30 minute ETA from our network provider to begin scrubbing traffic in a location with capacity."

    https://twitter.com/dnsimplestatus/status/539551209452232705

  • by kjttm on 12/1/14, 10:59 PM

    Does anyone have a simple explanation or link to an article / blog that explains the naked domain / ALIAS "problem" that DNSSimple solves? I recently set up DNS with DNSimple (due to nudging by Heroku) and am affected by this DDoS. I am still struggling to understand the exact nature of this issue. All of Heroku's documentation is pretty cryptic (to me):

    "Some DNS hosts provide a way to get CNAME-like functionality at the zone apex using a custom record type. " .. and then on to suggest DNSimple as their first suggestion.

  • by scott_karana on 12/1/14, 10:53 PM

    For those wondering about alternatives to ALIAS: if you use a www subdomain, then you can simply use CNAMEs. (Though the appearance is a matter of taste...)

    Google, Facebook, etc, all use this approach.

  • by shoxxx on 12/1/14, 10:19 PM

    Anyone switching from DNSimple? I really don't want to, but we've been down for almost 3 hours. I've seen chatter about Cloudfare and it looks pretty good, reviews?

  • by ataco on 12/2/14, 7:26 AM

    DNSimple is my registrar and (was my only) DNS provider. Now that they're back up I've exported the zone file and imported it to route 53 for redundancy in case this happens again. I also I updated the name servers in DNSimple to be 2 route 53, and 2 DNSimple, in that order. Is that the right way to do it? Does the order of the NS records matter? I set them up so that they're in the same order in both places.

  • by boopadoop on 12/4/14, 5:27 AM

    DNSimple says it was not a direct attack on them but rather domains being brought over by new customers. Does anyone know the actual target?

  • by brianarmstrong on 12/3/14, 7:26 PM

    I wrote a follow-up article about what we at Canopy.co learned from this incident. Check it out (this covers and expands on some of the ideas talked about here):

    https://medium.com/@brianarmstrong/youre-probably-doing-dns-...

  • by soci on 12/1/14, 10:51 PM

    Unfortunately, it's not the first time it happens, my app is down and customers unhappy.

    I always wonder, why is it that someone wants to attack a small company like DNSimple ? Is it that they were blackmailed and did not surrender to the criminals? If so, why would anyone be interested in blackmailing such a small company?

  • by anderly on 12/2/14, 1:03 PM

    You can use my cross-platform cli for dnsimple to export your zone files easily to txt or json format: https://www.npmjs.org/package/dnsimple-cli

    dnsimple domain record list example.com > example.txt

    OR

    dnsimple domain record list example.com --json > example.json

  • by stockkid on 12/1/14, 11:40 PM

    RubyGems.org and Travis-ci are down as a result of this! Not helping with my productivity this morning.

  • by beck5 on 12/1/14, 9:56 PM

    What are the recommended practices to prevent too much down time when your DNS provider goes down?

  • by pkfrank on 12/1/14, 9:11 PM

    Can anyone expand on what this means: "This attack is volumetric in nature." (?)