by robinhoodexe on 7/1/24, 8:40 AM with 338 comments
by FiloSottile on 7/1/24, 9:45 AM
When PerSourcePenalties are enabled, sshd(8) will monitor the exit
status of its child pre-auth session processes. Through the exit
status, it can observe situations where the session did not
authenticate as expected. These conditions include when the client
repeatedly attempted authentication unsucessfully (possibly indicating
an attack against one or more accounts, e.g. password guessing), or
when client behaviour caused sshd to crash (possibly indicating
attempts to exploit sshd).
When such a condition is observed, sshd will record a penalty of some
duration (e.g. 30 seconds) against the client's address.
It's not really a reversable patch that gives anything away to attackers: it changes the binary architecture in a way that has the side-effect of removing the specific vulnerability and also mitigates the whole exploit class, if I understand it correctly. Very clever.
by NelsonMinar on 7/1/24, 2:30 PM
> Successful exploitation has been demonstrated on 32-bit Linux/glibc systems with ASLR. Under lab conditions, the attack requires on average 6-8 hours of continuous connections up to the maximum the server will accept. Exploitation on 64-bit systems is believed to be possible but has not been demonstrated at this time. It's likely that these attacks will be improved upon.
by jamilbk on 7/1/24, 6:46 PM
void
sigdie(const char *fmt,...)
{
#ifdef DO_LOG_SAFE_IN_SIGHAND
va_list args;
va_start(args, fmt);
do_log(SYSLOG_LEVEL_FATAL, fmt, args);
va_end(args);
#endif
_exit(1);
}
void
sshsigdie(const char *file, const char *func, int line, const char *fmt, ...)
{
va_list args;
va_start(args, fmt);
sshlogv(file, func, line, 0, SYSLOG_LEVEL_FATAL, fmt, args);
va_end(args);
_exit(1);
}
What could have prevented this? More eyes on the pull request? It's wild that software nearly the entire world relies on for secure access is maintained by seemingly just two people [2].
[1] https://github.com/openssh/openssh-portable/commit/752250caa...
[2] https://github.com/openssh/openssh-portable/graphs/contribut...
by fanf2 on 7/1/24, 9:35 AM
This is actually an interesting variant of a signal race bug. The vulnerability report says, “OpenBSD is notably not vulnerable, because its SIGALRM handler calls syslog_r(), an async-signal-safer version of syslog() that was invented by OpenBSD in 2001.” So a signal-safety mitigation encouraged OpenBSD developers to put non-trivial code inside signal handlers, which becomes unsafe when ported to other systems. They would have avoided this bug if they had done one of their refactoring sweeps to minimize the amount of code in signal handlers, according to the usual wisdom and common unix code guidelines.
by qhwudbebd on 7/1/24, 11:37 AM
https://github.com/bminor/musl/blob/master/src/misc/syslog.c
Everything there is either on stack or in static variables protected from reentrancy by the lock. The {d,sn,vsn}printf() calls there don't allocate in musl, although they might in glibc. Have I missed anything here?
by cperciva on 7/1/24, 9:12 AM
https://www.freebsd.org/security/advisories/FreeBSD-SA-24:04...
by rfmoz on 7/1/24, 9:33 AM
> Finally, if sshd cannot be updated or recompiled, this signal handler race condition can be fixed by simply setting LoginGraceTime to 0 in the configuration file. This makes sshd vulnerable to a denial of service (the exhaustion of all MaxStartups connections), but it makes it safe from the remote code execution presented in this advisory.
Setting 'LoginGraceTime 0' in sshd_config file seems to mitigate the issue.
by 0x0 on 7/1/24, 9:06 AM
by CJefferson on 7/1/24, 9:55 AM
One thing which (as an independant person, who isn't doing any of the work!) is it often feels like in order to 'win', people are expected to find a full chain which gives them remote access, rather than just finding one issue, and getting it fixed / getting paid for it.
It feels to me like finding a single hole should be sufficient -- one memory corruption, one sandbox escape. Maybe at the moment there are just too many little issues, that you need a full end-to-end hack to really convince people to take you seriously, or pay out bounties?
by djmdjm on 7/1/24, 8:56 AM
Minimal patches for those can't/don't want to upgrade: https://marc.info/?l=oss-security&m=171982317624594&w=2
by acatton on 7/1/24, 1:34 PM
[1] https://www.tarsnap.com/spiped.html
by jesprenj on 7/1/24, 9:23 AM
by INTPenis on 7/1/24, 10:55 AM
They run sshd with the -D option already, logging everything to stdout and stderr, as their systemd already catches this output and sends it to journal for logging.
So I don't see anywhere they would be calling syslog, unless sshd does it on its own.
At most maybe add OPTIONS=-e into /etc/sysconfig/sshd.
by ttul on 7/1/24, 1:13 PM
1. It affects OpenSSH versions 8.5p1 to 9.7p1 on glibc-based Linux systems.
2. The exploit is not 100% reliable - it requires winning a race condition.
3. On a modern system (Debian 12.5.0 from 2024), the researchers estimate it takes: - ~3-4 hours on average to win the race condition - ~6-8 hours on average to obtain a remote root shell (due to ASLR)
4. It requires certain conditions: - The system must be using glibc (not other libc implementations) - 100 simultaneous SSH connections must be allowed (MaxStartups setting) - LoginGraceTime must be set to a non-zero value (default is 120 seconds)
5. The researchers demonstrated working exploits on i386 systems. They believe it's likely exploitable on amd64 systems as well, but hadn't completed that work yet.
6. It's been patched in OpenSSH 9.8p1 released in June 2024.
by megous on 7/1/24, 8:56 AM
In our experiments, it takes ~10,000 tries on average to win this race
condition, so ~3-4 hours with 100 connections (MaxStartups) accepted
per 120 seconds (LoginGraceTime). Ultimately, it takes ~6-8 hours on
average to obtain a remote root shell, because we can only guess the
glibc's address correctly half of the time (because of ASLR).
by nfriedly on 7/1/24, 1:54 PM
by letters90 on 7/1/24, 9:33 AM
Mitigate by using fail2ban?
Nice to see that Ubuntu isn't affected at all
by yjftsjthsd-h on 7/1/24, 11:44 AM
( https://www.openssh.com/txt/release-9.8 )
Darn - here I was hoping Alpine was properly immune, but it sounds more like "nobody's checked if it works on musl" at this point.
by lostmsu on 7/1/24, 1:16 PM
by ransom1538 on 7/2/24, 8:48 AM
1. Find things that are 0.0.0.0 port 22, example, https://gist.github.com/james-ransom/97e1c8596e28b9f759bac79...
2. Force them to the local network, gcloud compute firewall-rules update default-allow-ssh --source-ranges=10.0.0.0/8 --project=$i;
by gavinhoward on 7/1/24, 2:24 PM
by poikroequ on 7/1/24, 1:35 PM
by djernie on 7/1/24, 11:02 AM
by sharpshadow on 7/1/24, 7:13 PM
by nubinetwork on 7/1/24, 9:06 AM
by marcus0x62 on 7/1/24, 11:25 AM
https://archlinux.org/packages/core/x86_64/openssh/
edit be sure to manually restart sshd after upgrading; my systems fail during key exchange after package upgrade until restarting the sshd service:
% ssh -v 192.168.1.254
OpenSSH_9.8p1, OpenSSL 3.3.1 4 Jun 2024
... output elided ...
debug1: Local version string SSH-2.0-OpenSSH_9.8
kex_exchange_identification: read: Connection reset by peer
Connection reset by 192.168.1.254 port 22
by matthewcroughan on 7/1/24, 12:56 PM
by Wowfunhappy on 7/1/24, 10:44 PM
by ementally on 7/1/24, 6:39 PM
by madacol on 7/2/24, 12:12 PM
---
- OpenSSH < 4.4p1 is vulnerable to this signal handler race condition, if not backport-patched against CVE-2006-5051, or not patched against CVE-2008-4109, which was an incorrect fix for CVE-2006-5051;
- 4.4p1 <= OpenSSH < 8.5p1 is not vulnerable to this signal handler race condition (because the "#ifdef DO_LOG_SAFE_IN_SIGHAND" that was added to sigdie() by the patch for CVE-2006-5051 transformed this unsafe function into a safe _exit(1) call);
- 8.5p1 <= OpenSSH < 9.8p1 is vulnerable again to this signal handler race condition (because the "#ifdef DO_LOG_SAFE_IN_SIGHAND" was accidentally removed from sigdie()).
by Uptrenda on 7/2/24, 6:51 AM
edit: maybe i should add an iptable rule to only allow ssh from my IP.
by betaby on 7/1/24, 5:36 PM
by maxmalkav on 7/1/24, 3:29 PM
by programmer86 on 7/2/24, 3:22 PM
by thenickdude on 7/1/24, 8:03 PM
by rrix2 on 7/2/24, 3:30 AM
by MaximilianEmel on 7/1/24, 12:48 PM
by drsanthosh1997 on 7/2/24, 8:26 AM
by TacticalCoder on 7/1/24, 11:47 AM
I don't use port-knocking but I really just don't get all those saying: "It's security theater".
We had not one but two major OpenSSH "near fiasco" (this RCE and the xz lib thing) that were both rendered unusable for attackers by using port knocking.
To me port-knocking is not "security theater": it adds one layer of defense. It's defense-in-depth. Not theater.
And the port-knocking sequence doesn't have to be always the same: it can, say, change every 30 seconds, using TOTP style secret sequence generation.
How many exploits rendered cold dead in their tracks by port-knocking shall we need before people stop saying port-knocking is security theater?
Other measures do also help... Like restrictive firewalling rules, which many criticize as "it only helps keep the logs smaller": no, they don't just help keep the logs smaller. I'm whitelisting the three ISP's IP blocks anyone can reasonably be needing to SSH from: now the attacker needs not only the zero-day, but it also need to know he needs to be on one of those three ISPs' IPs.
The argument that consists in saying: "sshd is unexploitable, so nothing else must be done to protect the server" is...
Dead.
by nj5rq on 7/1/24, 11:07 AM
OpenBSD is notably not vulnerable, because its
SIGALRM handler calls syslog_r(), an async-signal-safer version of
syslog() that was invented by OpenBSD in 2001.
by j16sdiz on 7/1/24, 11:01 AM
by Sparkyte on 7/2/24, 4:20 AM
I kid, but really you probably shouldn't on Production. You should be exporting your logs and everything else. The host or VM bootstrapped golden images with everything as needed.
It is okay to start that way and figure out your enternals but that isn't for Production. Production is a locked down closed environment.
Recomment from another Hacker News post.
by sneak on 7/1/24, 3:07 PM