from Hacker News

io_uring_spawn: Launching New Processes with io_uring

by goranmoomin on 9/14/22, 12:52 PM with 41 comments

• by kyaghmour on 9/14/22, 3:14 PM

  I attended Josh's talk earlier this week at LPC. As I told Josh yesterday in the hallway track, I've been attending Linux conferences for 20+ years and often times some topics keep coming back and you can see the specialization. But in this case, I was completely blown away. This is a totally new approach to an old problem. I mean, fork() was invented in the '70's and aside from threads and clone(), in Linux, there really hasn't been any innovation in this space over the years. I look forward to seeing what else gets io_uring'ed following this model.

• by boffinAudio on 9/14/22, 1:23 PM

  This is really exciting work - it has the potential to make Linux software services way, way more efficient.

  I look forward to seeing the impact that this change has, for example, on future builds. A significant amount of time is spent at the beginning of a process spawn, setting things up - being able to batch things up without the process startup load, is going to make things much, much faster. Process pools? Yes please! :) I will definitely be a customer of posix_spawn() when its ready ..

  One has to wonder, though, given Josh' work with Rust, if any of this work is going to be done in Rust? One can dream ..

• by staticassertion on 9/14/22, 3:37 PM

  So, to my knowledge io_uring is a complete seccomp bypass (for all functionality it implements). Docker allows io_uring because "so far it isn't that big of a deal" (paraphrasing from Github issue 2 years ago). Given that io_uring now provides file IO and process execution... isn't it about time that someone starts thinking about the security implications? Are we going to just have a universal seccomp bypass for anyone who uses io_uring? How're the audit hooks coming along? Or LSM hooks?

  edit: K I've reached my HN rate limit again, thanks Dang.

  Anyway, here's a source for the Docker seccomp filter since someone replied inquiring.

  https://github.com/thaJeztah/docker/blob/master/profiles/sec...

• by the_duke on 9/14/22, 1:38 PM

  What kind of workloads spawn so many processes that saving microseconds becomes relevant?

  I guess shells (eg with xargs) would be one example.

  Edit: the talk mentions build systems as a motivating use case.

• by nos4A2 on 9/14/22, 3:10 PM

  The comparison does not seem to be apples to apples. The perf gain seems to come from bundling the fork and exec call allowing the kernel to do vfork like optimizations. I think a fair comparison would be against a new sys call that does fork_exec vs io_uring_spawn, and not fork + exec vs io_uring_spawn.

• by joeyh on 9/14/22, 3:18 PM

  The performance of this might be useful for a few things. The correctness seems more generally useful. The first two things I looked at that currently use vfork both go on to do things that are unsafe (according to the information in this talk anyway). I'll bet a substantial number of programs that use fork don't limit themselves to signal-safety(7) functions either.

• by ori_b on 9/14/22, 3:45 PM

  Does this matter when you have shared libraries? ld.so will be doing hundreds of syscalls at start regardless.

• by wmf on 9/14/22, 3:58 PM

  If I'm reading this right it should also allow memory overcommit to be disabled.

• by egberts1 on 9/14/22, 1:37 PM

  Reminds me of a single pit stop where a queue of race cars await the “go” words.

  Still need resources behind the pit stop, lots and lots of resources.