from Hacker News

Encrypted DNS and NTP = Deadlock

by cyounkins on 12/29/22, 8:14 PM with 72 comments

• by CaliforniaKarl on 12/29/22, 11:42 PM

  Assuming devices like this get their IP via DHCP, there is a solution that does not involve hard-coding IPs into software.

  DHCP option 42 (defined in RFC 2132) can be used to specify multiple NTP server IPv4 addresses.

  (There’s also DHCP option 4, but that’s used to specify the IP for the older RFC 868 time protocol.)

  DHCPv6 has option 31 for SNTP (via deprecated RFC 4075), and option 56 for NTP (via RFC 5908).

  So, that would probably be the best option: Get an NTP address from DHCP or DHCPv6, use that to set your clock, bring up DNS over TLS/HTTPS, re-configure NTP with your preferred source, re-sync your clock, and then continue booting!

• by CGamesPlay on 12/30/22, 6:29 AM

  I think there should be an option in TLS clients to ignore expiration times, explicitly for cases like this. All other validations are performed, just the not-before and not-after times are waived.

  Of all the validations, this one seems like the one that causes problems in edge cases most frequently. I'm definitely not saying that expiry times should be ignored by default; just that client should have the option to do it.

  This is obviously a problem for embedded devices, but it even goes to the browser level: when a certificate expired yesterday but is otherwise valid, I as a user want to be able to ignore that and only that error, particularly for a pinned certificate, since it's more likely to be incompetent sysadmin than malicious attacker.

• by jedisct1 on 12/29/22, 9:27 PM

  You can add IP addresses of the NTP servers being used in the captive-portals.txt file, which serves records even before the operating system considers a network interface as active.

  Or add cert_ignore_timestamp=true to the main configuration file. Initially, cert expiration won't be checked, but as soon as a DNS server will be reachable, this feature will automatically disable itself.

  This is for dnscrypt-proxy. Alternative clients may have something similar.

• by raggi on 12/30/22, 6:30 AM

  https://datatracker.ietf.org/doc/html/draft-ietf-ntp-roughti... roughtime has been in development since agl proposed it in 2016. It'd be nice to see it get over the line - until then tlsdate is easy to apply for approximate bootstrapping.

• by mvip on 12/30/22, 8:45 AM

  We've seen this (and similar TLS related issues) a fair bit at Screenly when working with Raspberry Pis.

  The best workaround that we've found is to use the date in a HTTP Header to set the initial time (if we detect this condition):

  $ curl -sI http://api.screenlyapp.com | grep Date

  Date: Fri, 30 Dec 2022 08:43:56 GMT

  With this set, you should be able to trigger the initial NTP service to start and set the date.

  There's a Rust library here that can parse these dates for you: https://docs.rs/httpdate/0.3.2/httpdate/

• by miyuru on 12/30/22, 5:13 AM

  > It would be great to see Google or Cloudflare use their infrastructure to provide anycasted NTP IP addresses.

  Google, Cloudflare and Facebook has vanity IPv6 address, pretty sure they are all static anycast IPs.

  time.google.com - 2001:4860:4806::

  time.cloudflare.com - 2606:4700:f1::123

  time.facebook.com - 2a03:2880:ff0c::123

• by dblitt on 12/30/22, 5:57 AM

  A similar issue happens when a laptop (with a bad battery) loses its time and can't connect to 802.1x WPA2 enterprise wifi.

  At my high school, we had laptop carts that were notorious for losing their time, and nobody could log in because they were bound to AD over wifi. The system was offline because it would reject the RADIUS certificate of the wifi network due to the time being wrong. We had to manually log in as local admin to change the time or plug them all into ethernet until they could connect to NTP.

• by LinuxBender on 12/29/22, 9:05 PM

  There may be a clunky work around, depending on what else is on the embedded devices. If it has curl, one could create a local host entry for a known site yes, bad practice I know and then curl --head over plain http to get the date header and use hwclock to bootstrap the system time prior to starting up NTP. Some NTP daemons also have a way to do something like this. Another method would be to bootstrap NTP with a few known dedicated public NTP servers that are in /etc/hosts then switch to the pool. These are all clunky methods but I have seen them used out of desperation for a myriad of "but you shouldn't do that" reasons. A cron job could check a management site for the latest configuration from a json or plain text file so that the device does not fall too far out of sync.

• by KirillPanov on 12/30/22, 9:54 AM

  Same problem with Wireguard and NTP.

  You can't (usefully) tunnel NTP inside of wireguard, because if your clock is wrong your peers won't talk to you anymore.

  This is my personal pet peeve.

  IMHO the wireguard handshake needs to be extended to allow one peer (the one that didn't reboot) to reply to a packet with a non-monotonically-increasing nonce with some signal saying "hey, here is the last nonce I got from you". Obviously this reply would be encrypted.

  Then hazmat-free hardware could use these replies to reset its nonce (for that particular peer only) if we haven't had a successful handshake with that peer since the last reboot and/or the system clock is implausible. Obviously this behavior would be off-by-default. I would enable it for my batteryless routers.

• by bobdvb on 1/3/23, 10:57 AM

  I think hard coding IPs is generally a bad idea, it might work for one or two users, but if this became standard practice then it would cause issues. I think it would be saner to say "if you don't have valid time (e.g. less than system/kernel build date) then don't use encrypted DNS.". Then NTP domains can be looked up, the answer will be correct enough to set a clock.

  Alternatively it would be good to use an anycast IP for NTP. This is normally a bad idea because it makes calculating skew hard/unreliable, but that really should just mean a poorly sync'ed clock. So set the Anycast clock to be an intentionally high/poor Stratum score, list this along with a DNS based address so it's used until the encrypted DNS can be resolved with a better Stratum score.

  So, Dear Akamai/Cloudflare/MANGA/etc. please provide a high stratum, Anycast address for basic, approximate NTP.

• by gonzo on 12/30/22, 7:44 AM

  Back in the day, Unix wrote the time of day in the superblock for the root fs before unmounting it and rebooting.

• by rsimmons on 12/30/22, 3:27 AM

  Can we change the title to Deadclock

• by tristor on 12/30/22, 5:50 AM

  > It would be great to see Google or Cloudflare use their infrastructure to provide anycasted NTP IP addresses.

  They do. Cloudflare does anyway. time.cloudflare.com is backed by a set of anycasted IPs covering around 275 POPs.

• by Joel_Mckay on 12/30/22, 7:52 AM

  In general, a time-travel sanity check at bootup is wise to handle when the RTC clock battery fails or is just installed (i.e. if the hardware jumps decades back into the past, than the local NTP/DNS/DHCP service is briefly paused while an abnormally large leap-forwards is forced from the time configuration commands. During this process, both client and server side SSL will usually be dropped due to the abnormality.)

  Rookie mistake, like not using UTC time on the servers. =)

• by thayne on 12/30/22, 7:40 AM

  Another potential way to work around this is to use unencrypted DNS for the NTP lookup only. In most cases you probably don't care about your lookup for an ntp server being confidential.

• by remram on 12/30/22, 6:27 PM

  I thought the title read "encrypted (DNS and NTP)", e.g. "encrypted DNS and encrypted NTP".

• by eniac111 on 12/30/22, 8:50 AM

  Happens to me several times a year. My PiHole is running on LXD :)

• by notwokeno on 1/3/23, 11:00 AM

  DNS is a public database. Encrypting it is a bit silly anyway IMO. Certainly between the recursive and authoritative resolvers.

• by exabrial on 12/30/22, 12:38 AM

  DNS over TCP/TLS is a stupid idea for a lot of reasons. First off, DNSSec already takes care of integrity protection. No need to re-implement the wheel by adding TLS on top of it… and let’s not forget we’re doing in 150 packets which one we previously did in about two packets.

  DNS needs to be connectionless; it’s building block protocol for TCP. DnsCurve is much closer to what we actually need.