from Hacker News

Docker Performance Evaluation Across Operating Systems

by drgo on 8/27/24, 5:26 PM with 37 comments

by zeehio on 8/27/24, 9:19 PM
> s. Upon analyzing the distribution of Docker Desktop for Windows and Docker Desktop for macOS, it was discovered that running the Docker environment on these requires a lightweight virtual machine that emulates the Linux system.
"it was discovered" as if the VM behind the docker engine running on those operating systems was hidden to the human race.
> The dynamite plots.
Please don't create dynamite plots. Use a box plot, or plot the dots of the different repetitions with some jitter to avoid overlap.
> The evaluation
As reported in other comments apparently OSX cheats when saying it's writing to disk. If the authors wanted to compare docker executions it would be useful for the discussion to see if the difference in performance was due to the docker overhead or to the operating system, by showing what's the write performance of a non-containerized program across operating systems.
In my opinion the article should not have been published as is. It should have been rejected or asked for major revisions. MPDI Applied Sciences is quite a bad journal already for "Computer Science Applications" https://www.scimagojr.com/journalsearch.php?q=21100829268&ti... so it's not surprising to see this published.
by stackskipton on 8/27/24, 9:37 PM
Ops person here, I'm confused what meaningful question this paper is supposed to answer?
Windows is suboptimal for running Linux containers? Sure, Docker on Windows is meant as development tool. If you have Windows Server but need to run Linux containers in production, you should run Hyper-V, install Linux and run container inside that VM.
MacOS has some funkiness for running containers as well that makes me think that's its disk operations are not truly backed. Also, Docker Desktop is again for Development, not actually running it in production since that would be really expensive server.
This entire paper could be a blog article. Did someone need to publish for some reason and threw this out there? What knowledge is being gained here? I WANT TO KNOW.
by drgo on 8/27/24, 5:26 PM
Abstract: Docker has gained significant popularity in recent years. With the introduction of Docker Desktop for Windows and macOS, there is a need to determine the impact of the operating system on the performance of the Docker platform. This paper aims to investigate the performance of Docker containers based on the operating system. One of the fundamental goals of this study is to conduct a comprehensive analysis of the Docker architecture. This technology utilizes Linux kernel virtualization mechanisms such as namespaces and cgroups. Upon analyzing the distribution of Docker Desktop for Windows and Docker Desktop for macOS, it was discovered that running the Docker environment on these requires a lightweight virtual machine that emulates the Linux system. This information suggests that the additional virtualization layer may hinder the performance of non-Linux operating systems hosting Docker containers. The paper presents a performance test of the Docker runtime on Linux, Microsoft Windows, and macOS. The test evaluated specific aspects of operating system performance on a MacBook computer with an ×86/64 processor architecture. The experiment carried out examined the performance in terms of CPU speed, I/O speed, and network throughput. This test measured the efficiency of software that utilizes various system resources.
by rdtsc on 8/27/24, 8:44 PM
> Ubuntu shows its fragility, especially in random writing. macOS outperforms Ubuntu by a substantial margin, exceeding its performance by a factor exceeding 40
They talk about this "fragility" like it's a common, known thing. I must be living in the woods, so to speak and feel silly. What's this known Ubuntu fragility they mention, anyone know?
by rlpb on 8/27/24, 9:18 PM
This "performance evaluation" is seriously flawed. I don't see how the results presented can be interpreted in any meaningful way.
> This barrier could be overcome with the help of the T2-Ubuntu [32] distribution, designed to be installed on Apple computers with T2-chipped.
That reference leads to https://github.com/t2linux/T2-Debian-and-Ubuntu-Kernel?tab=r..., which provides instructions on how to use a non-Ubuntu kernel to work around a hardware support issue on the particular hardware they used.
So they didn't actually test Ubuntu at all. They tested a third party produced kernel hack used to get support for their hardware. They then falsely assume that this is somehow representative of running Docker in...what? Production? Or for development purposes? They seem to have neglected to think about their use case. You'd think that the hardware they used for this test would be representative of a developer use case - but then why does performance matter?
If you consider container technology of the form that Docker uses, the kernel used is paramount for performance. Host OS userspace doesn't matter since it isn't active - the container userspace is provided by the container!
So not using an official Ubuntu kernel means that they didn't actually test anything provided by Ubuntu at all in practice. They've just tested whatever Linux kernel Docker Desktop uses inside its VMs on Windows and macOS against this Linux kernel hack running natively.
Their abstract also says:
> Upon analyzing the distribution of Docker Desktop for Windows and Docker Desktop for macOS, it was discovered that running the Docker environment on these requires a lightweight virtual machine that emulates the Linux system.
This is inherently known to anyone who can remotely claim to understand container technology. If this was a revelation to them, then this would explain why they think they tested Ubuntu when they didn't, and why they were unable to spot this flaw.
Disclosure: I'm biased in favour of Ubuntu. But I think the flaws I've described above stand for themselves.
by julian37 on 8/27/24, 8:08 PM
OrbStack is where it's at on macOS these days. It's so much better than Docker Desktop, it's not even funny. It's in a whole different league.
I'd take this article with a grain of salt when it doesn't even mention it, or any of the other alternatives.
by wolf550e on 8/27/24, 7:03 PM
The macOS IO benchmark and pgbench (also IO) makes me think macOS cheats somehow, for example it doesn't actually cause the SSD to write to flash when the app performs fsync, and some other "optimizations".
by asimpson on 8/27/24, 7:47 PM
Important caveat: "Apple MacBook Pro with Intel chip was used in tests that met all requirements specified in this section". What I really want to see is performance evaluations between current generation x86 Linux, x86 Windows, and ARM macOS.
by 3np on 8/28/24, 3:09 AM
They really should have run the benchmarks not only in Docker containers but also uncontainerized (as well as one or two KVM VM configurations) as baseline to approximate the overhead imposed by the Docker runtime itself. That would have made the article actually interesting.
Would also have been interesting to compare different runtimes (podman,crun,runc) and drivers.
I hope the authors follow up with a more fleshed out version because the chosen benchmarks and setup could make for some interesting results with some more rigorous methodology and analysis.
by brcmthrowaway on 8/27/24, 7:05 PM
I jusst used Docker for the first time - I can't believe I didn't know about it before. I thought the only way to run Linux on macOS was using some sort of Virtualbox-esque solution.
I just don't like how the containers are ephemeral.. I want to use it like a VM.
by jihadjihad on 8/27/24, 8:31 PM
Could use a [pdf] in the title if someone wants to add that
by 0xbadcafebee on 8/27/24, 8:38 PM
It would appear from this paper's conclusion that installing software with default settings may lead to poor performance.