from Hacker News

Avoiding Complexity with Systemd

by irl_ on 6/27/21, 7:14 AM with 347 comments

• by WesolyKubeczek on 6/27/21, 1:08 PM

  For me, systemd is the best thing since sliced bread.

  As a programmer, I now don't need to care about dropping privileges, managing logging, daemonization (the moment I need to do the double-fork dance again, chairs will be flying, I swear), dropping into a chroot, and do half-arsed guesses "is it up and running yet?" from a convoluted mess of shell code that looks like a bunch of hair stuck down a drain for a month.

  I just write an easy-to-debug program which I can launch from command line and see it run, and when I'm satisfied, a systemd unit from a cookie-cutter template is going to make it run. Service dependencies are now a breeze, too.

  If I need to limit resources, I can just declare the limits in the unit. If I want a custom networking/mount namespace, it's taken care of.

• by Dinux on 6/27/21, 9:18 AM

  I'm glad to see more people sway to systemd. Systemd is 10 years in the making and it was met with skepticism right from the first day. Some of that is now slowly changing with systemd being accepted in more and more distributions. Service and runlevel management wasn't any better in the sysv era, nor were any of the multitude of custom start and boot scripts.

  I remember when it took multiple days testing the configuration on different distributions, editions and versions to just get a single daemon process to start without failure. Then do the whole thing over again because the Debian based distros did not use the RedHat startup tools, different (or no) runlevel configurations, different service binders, different NTP daemon, different terminal deamon, etc.. And of course the French customers wanted the service to work with Mandriva, the German customers want SUSE support with extended security options like dropping permissions after boot.

  Just like the article mentions you can define a portable service model with security and failure handling built in. There wasn't even anything that came close back in the day. Systemd may not have been built with the Unix philosophy in mind, but at some point that becomes a secondary concern.

  Systemd unifies all systemd resources in units which work anywhere, its expandable and extendable, user friendly, allows for remote monitoring etc.

• by tyingq on 6/27/21, 11:30 AM

  My frustration with systemd is that it creeps out over time into other functionality, and that it's difficult to find the right documentation.

  Like when logind was changed to kill background processes when you log out, by making KillUserProcesses=yes the default. Some Linux distros left that as is, others overrode it in /etc/systemd/logind.conf. So, figuring out what was happening, and how to fix it, was confusing. I had no idea it would have been logind doing that.

  Similar for changes introduced with systemd-resolved.

• by knorker on 6/27/21, 11:57 AM

  My problem with systemd is that it's just so poorly written.

  The ideas are not inherently bad. But they're not thought through, and the implementation is pure garbage.

  Like taking the most stable software in the world[1], and going "nah, I'll just replace it with proof-of-concept code, leaving a TODO for error handling. It'll be fine.".

  And then the "awesomeness" of placing configuration files whereever the fuck you want. Like /lib. Yes, "lib" sounds like "this is where you place settings file in Unix. At least there's no other central place to put settings".

  [1] Yes, slight hyperbole. But I've not had Linux init crash since the mid-90s, pre-systemd

• by stefanha on 6/27/21, 11:40 AM

  Recently I have been wondering if systemd solves problems that are becoming less and less relevant to developers. New services are often deployed as containers.

  While systemd has a bunch of container-related functionality, it does not integrate well into the Kubernetes or even Docker workflow. It's used very little in those environments.

  If you are building CoreOS or NixOS system images, or traditional Linux system services, then systemd matters. But I think way more services are being built for the container world where these problems are solved differently.

  For example, the TLS configuration can be handled with common container patterns. The author's startup example would translate more easily to a full-blown Kubernetes environment once the VC funding hits their bank account if they had used containers from the start instead of first writing the service for systemd.

  It's a shame because systemd is very powerful and I've enjoyed using it.

• by candiddevmike on 6/27/21, 12:48 PM

  Only complaint I have about systemd is their docs aren't versioned, so it's difficult to see if the functionality you're looking for is available in the version you're running. Most of the time I have to Ctrl F through the NEWS doc in their repo.

• by turminal on 6/27/21, 10:00 AM

  The title is an oxymoron.

  Putting a huge complex piece of software between yourself and "complexity" doesn't make the system less complex.

• by whateveracct on 6/27/21, 8:16 AM

  Use NixOS and you'll love systemd.

  You'll be defining your own systemd units with ease.

  systemd to you will be journalctl and systemctl. So pretty good.

• by folex on 6/27/21, 8:53 AM

  I just learned about S6 recently. It simplified daemon supervision for me a lot.

• by tbrock on 6/27/21, 10:48 AM

  This should be part of the manual. Ive tried multiple times to understand systemd deeper than a service restart here and looking at logs for a unit there to no avail.

• by kkirsche on 6/27/21, 11:06 AM

  As someone who wants to learn systemd, even just to understand what I’m stuck with and why it was made, this is the best introduction I’ve found. Does anyone have other resources to help beginners learn how to use systemd efficiently and it’s relationship to Docker?

• by djhworld on 6/27/21, 9:15 AM

  Really enjoyed reading this article.

  The LoadCredentials thing reminds me of configmaps in K8S, is there a more general thing in systemd e.g LoadConfig

• by CGamesPlay on 6/27/21, 5:55 PM

  In the TLS example, how does systemd handle certificate rotation? Are the certificate files symlinks, hard links, or copies? If hard links or copies, presumably my service will need to be signaled or restarted to get the new certificate; does systemd do this?

• by kemotep on 6/27/21, 12:08 PM

  Whenever these discussions of systemd come up, I am reminded of this talk[0].

  It will be interesting to see if one day a replacement for systemd comes along and people who once championed systemd will begin to use the arguments the people who do not prefer systemd use to defend their choices for not wanting to use the next init system manager.

  [0]: https://youtu.be/o_AIw9bGogo

• by sgt on 6/27/21, 1:39 PM

  If you are looking to run your containers in a very light weight way and also easily understood, you can use systemd for this and use tips from this article. You will have to do the orchestration yourself so I think it would have to be more suitable for very simple deployments with small teams and / or part time projects.

• by PaulHoule on 6/27/21, 5:49 PM

  I don’t see it as a systemd-ism but that the UNIX way is maturing.

  Passing an arbitrary fd or socket from one process to another solves many problems and we are in the habit of doing it now.

• by Dah00n on 6/27/21, 9:17 AM

  I read that as meaning "avoid systemd's complexity" as opposed to "avoid complexity by doing so and so using systemd". Oh well.

• by robertlagrant on 6/27/21, 8:45 PM

  To;Dr: good lord it's hard to avoid complexity when you're not using Docker to run your app.

• by teknopaul on 6/27/21, 3:55 PM

  Trying to avoid complexity by being dependent on something horribly complex is not going to work.

• by bitwize on 6/27/21, 10:19 AM

  You incur complexity with systemd. You avoid complexity with runit.

• by baybal2 on 6/27/21, 12:41 PM

  I want to mention connman.

  Does everything you expect of single programmlet to manage your NTP, resolve.conf, DNS caching, mDNS, network devices, and etc.

  Importantly, it weights only 1/100 of SystemD

• by einpoklum on 6/27/21, 2:11 PM

  The title is quite triggering. As I see it, systemd itself is very complex. So, one might say "Guaranteeing complexity with systemd", though possibly being able to ignore the complexity, usually.

• by imiric on 6/27/21, 10:53 AM

  Apologies, I was also triggered by the title. :)

  > systemd provides ways to restrict the parts of the filesystem the service can see.

  So like chroot and namespaces? Why do I have to depend on systemd when these are native features provided by Linux?

  So systemd provides a friendlier abstraction of these concepts. Great, but so do Docker and Podman and many other tools that can actually be installed without taking over the rest of the system.

  Having your application actually use systemd libraries further increases this dependency and makes it no longer usable but on a subset of Linux machines. This would be fine for some controlled production deployment, but is awful for usability and adoption.

• by TruthWillHurt on 6/27/21, 11:33 AM

  I'd suggest you stear away from using systemd and a server to launch your startup.

  While this is a good writeup, and you end up with a service, you still need to manage a machine with all risks involved - server reboots, updates, networking etc.

  AWS Fargate, or the new App Runner will manage a container almost hassle-free