by sublimefire on 3/7/22, 1:43 PM
An example that needs to be in the textbooks. A detailed explanation and a timeline along with the code snippets. It succinctly shows you the complexities involved. Kudos to Max for putting it all into the post.
> Blaming the Linux kernel (i.e. somebody else’s code) for data corruption must be the last resort.
^^^ I can only image the stress levels at this point.
by mltony on 3/7/22, 9:54 PM
10 years ago I found even more outrageous bug in Windows 8.
I was working in MSFT back than and I was writing a tool that produced 10 GB of data in TSV format , that I wanted to stream into gzip so that later this file would be sent over the network. When the other side received the file they would gunzip it successfully, but inside there would be mostly correct TSV data with some chunks of random binary garbage. Turned out that pipe operator was somehow causing this.
As a responsible citizen I tried to report it to the right team and there I ran into problems. Apparently no one in Windows wants to deal with bugs. IIt was ridiculously hard to report this while being an employee, I can't imagine anyone being able to report similar bugs from outside. And even though I reported that bug I saw no activity in it when I was leaving the company.
However I just tried to quickly reproduce it on Windows 10 and it wouldn't reproduce. Maybe I forgot some details of that bug or maybe indeed they fixed this by now.
by staticassertion on 3/7/22, 2:26 PM
Another example of a vulnerability that is purposefully obfuscated in the commit log. It is an insane practice that needs to die. The Linux kernel maintainers have been doing this for decades and it's now a standard practice for upstream.
This gives attackers an advantage (they are incentivized to read commits and can easily see the vuln) and defenders a huge disadvantage. Now I have to rush to patch whereas attackers have had this entire time to build their POCs and exploit systems.
End this ridiculous practice.
by blinkingled on 3/7/22, 1:23 PM
The offending commit was authored by Christoph Hellwig and possibly reviewed by Al Viro both of whom combined are close to 100% of Linux filesystems and VFS knowledge. Point being with the level of complexity you're just going to live the fact that they'll always be bugs.
VFS/Page Cache/FS layers represent incredible complexity and cross dependencies - but the good news is code is very mature by now and should not see changes like this too often.
by cryptonector on 3/7/22, 7:02 PM
There was a never-shipped bug in Solaris back around.. I want to say 2006? I don't remember exactly when, but there was a bug where block writes in a socketpair pipe could get swapped. I ended up writing a program that wrote entire blocks where each block was a repeated block counter, that way I could look for swapped blocks, and then also use that for the bug report. The application that was bit hard by this bug was ssh.
Writing [repeated, if needed] monotonically increasing counters like this is a really good testing technique.
by gaius_baltar on 3/7/22, 1:27 PM
Fix was already merged to Android, however, there are millions of devices that will never be updated. The nice question: can this be used for temp-rooting? Vulnerabilities can be a blessing sometimes...
by parmezan on 3/7/22, 1:15 PM
It has been less than a month after fixes emerged for kernels and your PoC exploit has already been released into the public. Should you not have waited at least a bit longer (for example 2 months) before disclosing this vulnerability so that people/companies can keep up with patching? Don't they need more time to patch their servers and legacy etc before this becomes yet another log4j exploitation fest? That is if this really is the new dirty cow vuln.
I get responsible disclosure is important, but should we not give people some more opportunity to patch, which will always take some time?
Just curious.
Also, nice work and interesting find!
by legalcorrection on 3/8/22, 6:21 PM
>
Let me briefly introduce how our log server works: In the CM4all hosting environment, all web servers (running our custom open source HTTP server) send UDP multicast datagrams with metadata about each HTTP request. These are received by the log servers running Pond, our custom open source in-memory database. A nightly job splits all access logs of the previous day into one per hosted web site, each compressed with zlib.Via HTTP, all access logs of a month can be downloaded as a single .gz file. Using a trick (which involves Z_SYNC_FLUSH), we can just concatenate all gzipped daily log files without having to decompress and recompress them, which means this HTTP request consumes nearly no CPU. Memory bandwidth is saved by employing the splice() system call to feed data directly from the hard disk into the HTTP connection, without passing the kernel/userspace boundary (“zero-copy”).
Windows users can’t handle .gz files, but everybody can extract ZIP files. A ZIP file is just a container for .gz files, so we could use the same method to generate ZIP files on-the-fly; all we needed to do was send a ZIP header first, then concatenate all .gz file contents as usual, followed by the central directory (another kind of header).
Just want to say, these people are running a pretty impressive operation. Very thoroughly engineered system they have there.
by itvision on 3/7/22, 2:36 PM
This if f*cking scary. Such a simple code, so dangerous and it works. You can trivially add an extra root user via /etc/{passwd|shadow}. There are tons of options how to p0wn a system.
Please update your devices ASAP!
by jesprenj on 3/7/22, 12:44 PM
This affects kernels from 5.8 and was fixed in 5.16.11, 5.15.25 and 5.10.102. Exploit code is public and available on the linked page.
by bananabiscuit on 3/7/22, 1:26 PM
I’m curious how git bisect was applied here. Wouldn’t you have to compile the whole kernel somehow and then run your test program using that kernel? Is that really what was done here?
by MayeulC on 3/7/22, 1:21 PM
Wouldn't this allow modifying a cached version of /sbin/su to nop the password check? This seems really easy to exploit for privilege escalation.
by nickelpro on 3/7/22, 1:05 PM
Excellent work and excellent write up Max. A feather in your cap to be proud of for sure.
by reasonabl_human on 3/7/22, 4:29 PM
Crazy. Just successfully pwnd my homelab box in the garage..
Exciting for the implications of opening many locked down consumer devices out there.
Nightmare for the greater cyber sec world...
by jwilk on 3/7/22, 1:46 PM
> A ZIP file is just a container for .gz files
That doesn't sound right.
by egberts1 on 3/8/22, 3:59 AM
Reminds me of SunOS 4.1.3 where you simply type in ‘+’ about 127 times at the “Login:” prompt and PRESTO-CHANGO … you get a root shell prompt.
by orwin on 3/7/22, 3:10 PM
Does this have a cvss yet? It seems really powerful and easy to exploit. And by easy to exploit I'm talking beginner CTF easy.
by DonHopkins on 3/7/22, 8:30 PM
Once I fell victim to The Dirty Bong Vulnerability, when the cat knocked the bong over onto my Dell laptop's keyboard. Fortunately I had the extended warranty, and the nice repairwoman just smelled it, laughed at me, and cheerfully replaced the keyboard for free. No way Apple would have ever done that.
by egberts1 on 3/7/22, 12:55 PM
Extreme Debugger Par Excellence!
What a supérioritégrandeur!
by cookiengineer on 3/7/22, 8:39 PM
Amazing write-up! This is a super example of a responsible disclosure.
I mean, compiling 17 kernels alone takes so long that most people would've given up in between.
by qwertox on 3/7/22, 5:56 PM
What a poster child. Deserves some kind of award.
by Dowwie on 3/7/22, 7:29 PM
>Memory bandwidth is saved by employing the splice() system call to feed data directly from the hard disk into the HTTP connection, without passing the kernel/userspace boundary (“zero-copy”).
What are the memory savings of this splicing approach as compared to streaming [through userspace]?
by abofh on 3/7/22, 12:52 PM
Wow, awesome debugging - very impressed.
by AviationAtom on 3/7/22, 8:51 PM
Since so many distros seem to lag a good ways behind on packages, and this vulnerability (in it's easiest exploited form) was introduced in kernel 5.8, it would seem a fair amount of Linux installs wouldn't actually be vulnerable to this. Is that somewhat correct?
by piratejon on 3/7/22, 1:08 PM
Wow, almost 10 months from the first reported file corruption until identification as an exploitable bug.
by aetherspawn on 3/7/22, 9:53 PM
The sort of bug that could have been caught by unit tests I suppose.
by bill_mcgonigle on 3/8/22, 5:14 PM
Thank you for not selling this to the "industry"!
by dcow on 3/8/22, 2:01 AM
The sample code does not demonstrate how to get the page to be flagged dirty so that the kernel actually writes it back to disk. Did I miss something?
by ZYinMD on 3/7/22, 3:18 PM
I think you're quite gifted in story telling, you could be thriller book writer.
by alanbernstein on 3/7/22, 5:20 PM
Dirty pipe.. how about "sewerbleed"?
by db48x on 3/7/22, 1:13 PM
C needs to die. Pro tip for language designers: require all fields to be initialized any time an object is created.
Really impressive debugging too.
by kgraves on 3/7/22, 1:04 PM
Google's Fuchsia/Zircon cannot come fast enough.