from Hacker News

Going in circles without a real-time clock

by X-Cubed on 4/11/24, 4:22 AM with 153 comments

  • by lifthrasiir on 4/11/24, 5:43 AM

    > By "everything else", I also mean WireGuard. Did you know that if your machine gets far enough out of sync, that'll stop working, too? I had no idea that it apparently includes time in its crypto stuff, but what other explanation is there?

    WireGuard only requires a monotonic clock, because it periodically rotates keys to provide the forward secrecy. Peer clocks are otherwise not required to be synchronized [1]. I guess the clock had a higher rate than usual and it couldn't be corrected due to the lack of RTC?

    [1] https://www.wireguard.com/papers/wireguard.pdf#page=7 "In fact, it does not even have to be an accurate timestamp; it simply must be a per-peer monotonically increasing 96-bit number."

  • by gorkish on 4/11/24, 5:13 PM

    The Pi ecosystem is coming apart at the seams. Talk about falling off a pedestal.

    By the time you buy everything you need to make a Pi 5 even remotely usable and reliable today, you'll have spent as much as a small form factor PC. What do you need?

      1. Special power supply - Yes, it's USB-C but it requires a high amperage 5V supply instead of accepting a higher voltage like most USB-C stuff
  2. Active cooler - Pi 5 throttles immediately without it; it's not an optional component
  3. NVMe Expansion HAT - Seriously why the fiddly little PCIe header? Put an NVMe slot on the bottom and let people adapt *that* connector
  4. RTC Battery - At least put a damn capacitor on the board
  5. Enclosure - I'm halfway expecting the pi foundation to print some dashed cut lines on the cardboard box so they can say it comes with a case =]
    At the end of this you get a system that can run ... raspbian

    Not like the Pi Foundation couldn't have contributed to uboot or the upstream kernel or anything before thier hardware was released. Pretty much nothing released for "Raspberry Pi" runs on a Pi 5 at this point in time.

    They better try to salvage something before putting out the expected CM5, but I'm not expecting much.

  • by mplaisier on 4/11/24, 7:14 AM

    I ran into a similar problem on a Raspberry Pi, where my program would ignore data because it was deemed too old. It turns out a Pi uses fake-hwclock. This tool stores a timestamp regularly in a file. On boot the Pi reads from that file to set the time. When the Pi has been turned off for longer then the initial time can be hours or days out of sync. The Pi then takes a couple of seconds to obtain a valid time via NTP, but my program ran quickly enough to work with the old timestamp. After a couple of minutes it would typically work again.

  • by zokier on 4/11/24, 7:32 AM

    It's unfortunate that using DHCP to get rough time isn't more popular, especially for devices without RTC. Sure DHCP can be spoofed and should be considered more of a last resort, but it sure would be better than nothing and ending up in this kind of a situation where nothing works anymore.

    Previous discussion on similar thing: "Encrypted DNS + NTP = Deadlock" https://news.ycombinator.com/item?id=34177331

  • by riobard on 4/11/24, 6:44 AM

    Speaking of RTC battery, I've recently come to the realization that I have to make sure that BIOS battery is not absolutely dead in always-on PC boxes.

    Background: I use an x86 box as home router. I've changed the configuration in the BIOS that it should automatically boot up on power. However if the BIOS battery is dead, the config will be lost and it will revert to default settings, which is not to boot on power.

    But there is no way to know how much juice is left in the BIOS battery, thus there is no way to issue warnings that the battery should be replaced soon. If there is power loss AND the battery is dead, when the power comes back up again, the router will just stay off indefinitely until it's manually turned on.

    That's when I realized why most consumer routers do not feature RTC nor do they require batteries.

  • by CaliforniaKarl on 4/11/24, 7:32 AM

    I know I’ve brought this up before, but I can’t remember exactly where.

    It would be nice if DHCP could provide the current time as one of the pieces of information it provides with a lease. If you’re already trusting it for your IP address, you might as well trust it with the current UTC time: The current 64-bit timestamp would be enough to get timestamp-sensitive things working.

  • by lightswitch05 on 4/11/24, 10:50 AM

    I had this happen before. I used it as an excuse to learn how to setup NTP with a GPS receiver. I made a little blog on it if anyone is interested in the results. Be sure to click the sandwich menu for some real-time data: https://www.developerdan.com/ntp/

  • by shawabawa3 on 4/11/24, 9:44 AM

    I've personally had so many issues with SD card corruption on raspberry pi's that I've decided it was a mistake to even allow SD card disks

    I only ever use USB storage on my pi's now and not had a single issue since

  • by cbhl on 4/11/24, 4:23 PM

    I think the last paragraph was my favorite part of this article.

    For those of you who didn't read the fine article, I've quoted it here:

    > As usual, this post is not a request for THE ONE to show up. If you are THE ONE, you don't make mistakes. We know. Shut up and go away.

  • by smashed on 4/11/24, 1:19 PM

    > I figured they must be running DNSSEC on that zone (or some part of it), and it must have a "not-before" constraint

    Since clients will attempt to resolve ntp.org in order to actually sync their clock, there is a good probability that some clients will be way off.

    Enabling dnssec on that zone was probably not without important drawbacks? I wonder if the operators thought about that potential pitfall. Seems like they might be doing a disservice to their core mission of allowing devices to sync their clock.

  • by raggi on 4/11/24, 6:35 PM

    If you don't have an RTC I'd recommend having a tlsdate with some bounding heuristics to prevent extreme clock fixation from a mitm. You can relatively cheaply hit a large number of public servers that are likely to have good times available over TLS and trust the common result. You validate the certs without considering the notbefore stamps and then if you're feeling aggressive validate them after you've managed to approximate a date from the cohort. I know there are commercial packages that do this, I'm not sure about OSS ones.

    Roughtime would be far better, but essentially there's no broad deployment of it yet.

    Ideally something good would be picked by Raspbian and delivered in the distro as standard.

  • by the__alchemist on 4/11/24, 2:20 PM

    It's surprising that it was left out. Ie, the workaround makes sense, but this is a peripheral that is sometimes built into MCUs themselves. For example, most (all?) STM32s include an RTC... with the caveat that if you are using them for canonical use cases, you will need external hardware in the form of a dedicated 32kHz oscillator, and possibly a battery.

    For a lot of micro-controller timing uses, they aren't required; for example, hardware timers based on the primary clock source, or the Cortex-M systick, but for maintaining accurate dates and times over long periods, the RTC is the right tool. It can also output the dates and times in a convenient format as well, as long as you find named register fields convenient!

  • by hcfman on 4/11/24, 8:35 AM

    Install sbts-aru as a base system and it will keep your Pi accuracy to sub-microsecond error.

    https://github.com/hcfman/sbts-aru

    Problem solved.

  • by Joel_Mckay on 4/11/24, 7:15 AM

    If you are running Bind or SSL/x.509 certs, than an RTC is pretty much a requirement on the raspberry pi. Network time leaps over NTP even when working (first big leap was allowed etc.) can still bork a lot of services.

    Local stratum 1 GPS time (even with PPS pin) servers based on Pi's can suffer similar issues when they lose satellite lock (weather etc.)

    Have a great day =)

  • by benmmurphy on 4/11/24, 1:17 PM

    I’m surprised tptacek hasn’t shown up to defend DNSSEC from this libel

  • by cjbillington on 4/11/24, 7:59 AM

    The wireguard problem is a pain in the neck. Even if you're happy for your system to not have a realtime clock, when it does come online you'd want wireguard to not start until after the clock has synced to a timeserver. The below is what I'm doing at the moment, but I can't say I'm sure it's working - haven't seen wireguard start before the time is synced since doing it, but it could be probabilistic:

        systemctl enable systemd-time-wait-sync.service
    mkdir -p /etc/systemd/system/wg-quick@wg0.service.d/
    echo "[Unit]
    After=time-sync.target
    Wants=time-sync.target
    " > /etc/systemd/system/wg-quick@wg0.service.d/override.conf
    I'd be interested if anyone could let me know if they think this is likely to be achieving what I want or not.

  • by wodenokoto on 4/11/24, 6:03 AM

    What is she referring to by “THE ONE”?

  • by malivvan on 4/11/24, 1:08 PM

    Unfortunately a Raspberry Pi is a bit ill suited for production environments. Id recommend an RTC module. Otherwise this might be helpful: https://github.com/hcfman/sbts-aru

  • by silasdavis on 4/11/24, 8:18 AM

    Tangential: I have a pi 4 running openwrt and a wireguard interface that is routed over the WiFi access point. If you connect to this network it's as if you are connecting in another country. I have outbound traffic only going over wireguard. If wireguard is down then no internet (to avoid any leakage). However if pi loses power the clock resets, wireguard can't handshake with that much clock skew, NTP can't connect without wireguard. So I need to manually update clock after a power failure.

    I guess I need requests to the NTP server to go over WAN directly. Always seemed like a hassle to get this to work with openwrt zones and stuff. My eternal gratitude to anyone who shoves me in the right direction...

  • by dclowd9901 on 4/11/24, 4:52 PM

    I’m not sure this is related or not but if there’s one thing I’ve learned about complex objects (engines, computers, software, sewing machines, etc), it’s that every _custom_ thing you do to it has some kind of order of magnitude impact on the overall complexity of the machine, and thus, you should avoid customizing it. At least if your primary concern is usability vs solving specific problems.

    It’s definitely served me well in a wide variety of disciplines.

  • by champtar on 4/11/24, 5:35 PM

    In OpenWrt (Linux distro for routers that often don't have RTC) dnsmasq starts with `--dnssec-no-timecheck` until the ntp client gets a first sync (https://github.com/openwrt/openwrt/commit/5acfe55d7139a52941...)

  • by aimor on 4/11/24, 2:36 PM

    Two nights ago I rebooted my router and it wasn't able to update its clock. The logs were filled with "ntp: start NTP update" every 5 seconds.

    Well, it was something related to using AdGuard's (94.140.14.14) DNS and pool.ntp.org. I added Google's (8.8.8.8) to resolv.conf temporarily to get the clock synced. But I'm still not sure what the root cause was.

  • by doubloon on 4/11/24, 12:30 PM

    Its weird that people work OK without knowing exactly what time it is. Sure it helps us but if a person lost track of the date for a few days in a city they would still be able to perform basic functions. We only have had timekeeping devices for a tiny part of our history but we invented alot of stuff without precise synchrony.

  • by jlubawy on 4/11/24, 6:30 AM

    My raspberry pi is also currently dead, something it didn't like about updating the OS caused it to be bricked. All updates were done within the OS, so didn't expect anything bad to happen.

    I could reflash the OS, but I also "don't have the time". I'll just use my laptop until I do have the time someday to resurrect it.

  • by cyounkins on 4/11/24, 8:06 PM

  • by the-kenny on 4/11/24, 12:24 PM

    Note that the Pi 5 has a real time clock - you just need to add a small battery and it will keep the time when powered off: https://picockpit.com/raspberry-pi/raspberry-pi-5-has-a-real...

  • by koala_man on 4/11/24, 5:42 AM

    tl;dr: machine is unable synchronize its clock because it won't trust the NTP server's DNSSEC certificate because its clock isn't synchronized. Oof.

  • by M95D on 4/11/24, 6:14 AM

    RTC module for Raspberry is ~ $1.