by charlieirish on 7/25/24, 10:07 AM with 279 comments
by tuetuopay on 7/25/24, 12:03 PM
OVH was held liable because of the data loss, not for the service interruption. Data loss is something irremediable, permanent, definitive. Some businesses were basically ruined from this incident because they had no more data to operate. To add insult to injury, they sold offsite backups in the datacenter literally meters away. A service interruption, well, shit happens, and this is handled by SLA contracts that both parties agree to. You don't ruin a business (read: close a company) for a few days of outage.
I doubt CrowdStrike will be held liable for much; from corporations at least. They cannot repay the damage done, or they close the door. The healthcare sector is another beast, but I think it will come to more regulations for critical entities.
by itunpredictable on 7/25/24, 11:35 AM
by siva7 on 7/25/24, 10:38 AM
by lordnacho on 7/25/24, 10:41 AM
I wonder how this kind of thing is organised, since there's all these jurisdictions.
by Retr0id on 7/25/24, 11:45 AM
In the OVH case, their backup system (as a whole) failed. Many customers were left with 0 data, and per the article "the court ruled the OVH backup service was not operated to a reasonable standard and failed at its purpose".
Meanwhile CrowdStrike "just" crashed their customer's kernels, for a duration of about 1 hour (during which they were 100% safe from cyber attacks!). Any remaining delays getting systems back online were (in my view) due to customers not having good enough disaster recovery plans. There's certainly grounds to argue that CrowdStrike's software was "not to a reasonable standard", but the first-order impacts (a software crash) are of a very different magnitude to permanently losing all data in a literal ball of fire (as in the OVH case).
Software crashes all the time. For better or for worse, we treat software bugs as an inevitability in most industries (there are exceptions, of course). While software bugs are the "fault" of the software vendor, the job of mitigating the impacts thereof lies with the people deploying it. The only thing that makes the CrowdStrike case newsworthy, compared to all the other software crashes that happen on a daily basis, is that CrowdStrike's many customers had inserted their software into many critical pathways.
CrowdStrike sells a playing card, and customers collectively built a house with them.
(P.S. Don't treat this as a defense of CrowdStrike. I think their software sucks and was developed sloppily. I think they should face consequences for their sloppiness, I just don't think they will, under current legal frameworks. At best, maybe people will vote with their wallets, going forwards.)
by dotancohen on 7/25/24, 10:50 AM
by bennyelv on 7/25/24, 10:45 AM
My understanding of law is generally UK based, but I'm not aware of legislation what would supersede a contract term limiting liability when the event that created the liability was one of general diligence/competence in carrying out the contract rather than relating to health and safety or some other area that is heavily legislated.
For that reason I'm unconvinced on the article's statement that this isn't just a "French Legal System" thing and that the same kind of judgement might be made in other jurisdictions.
by dathinab on 7/25/24, 12:21 PM
most(all?) EU have laws which limit how much you can opt out of liability _no matter what you write into a contract_
while I'm not sure about the exact boundaries per country but I'm pretty sure that at least all hospitals, emergency call services etc. can sue for a non-negligible part of the damages that outage caused directly
private people which where harmed by not getting operations done in time most likely can also sue them for the full damages caused to them (through it's hard to assess the damages and it might need to be indirectly by suing the hospital and the hospital sues for more damages)
what you likely will not be able to sue for is the lost opportunity cost, the man power needed to fix it etc.
also my guess is that for a lot of cases which are not as sever as human damages or as indirect as lost opportunity cost a huge factor will depend on the degree of negligence judges believe happened. And here "negligence" isn't limited to the specific change which caused the bug but also if they kept they due diligence in choices of tooling, approaches, business processes etc. to reasonable minimize the risk. (like e.g. was their way of parsing configs inadequate/did it follow industry best practices (IMHO it doesn't seem so), or was it adequate to mark the driver as required to allow boot (else windows would have auto disabled it and then restarted) etc.)
by MaximilianEmel on 7/25/24, 10:44 AM
I assume the year was meant to be 2024.
by spotirca on 7/25/24, 10:51 AM
Is there a link with this incident?
by notepad0x90 on 7/25/24, 11:51 AM
I think if the total liability for Crowdstrike is less than a few years worth of revenue, they'll come out unscathed because as I understand, they are still not profitable, their valuation is purely on speculation on future revenue. Their biggest paying customers still care a lot about getting compromised, it isn't just a box checking exercise like many have suggested.
by honzaik on 7/25/24, 10:40 AM
by pjmlp on 7/25/24, 11:17 AM
by anonu on 7/25/24, 11:40 AM
And if France comes down hard on them, they may simply not do business in France.
by classified on 7/25/24, 11:00 AM
by 627467 on 7/25/24, 2:02 PM
by wjnc on 7/25/24, 10:53 AM
From OP, in the OVH-case liability seems to override the contract / waivers when OVH was both the storage And backup provider and did not actively underline that this solution is suboptimal, in a situation where multiple data centers are physically very close. That's a chain of evidence.
For CrowdStrike, it is clear that the offering is to more mature counter parties (thus raising the B2B standard of evidence) and that CrowdStrike very essentially did not do / support staging, whatever. This is indeed bad industry practice, but one that can thought to be explicit from the start of the agreement. At least in my locale you either make explicit agreements OR industry standards are leading. We do not do industry standard X is pretty clear. Read the list in OP, replace CrowdStrike with Microsoft and then think of the international liability cases you've heard from where Microsoft was found liable for downtime, hacks and other issues.
Look, liabilities will always arise in such situations. But I expect only minor liabilities will arise. Mostly (AFAIK IANAL) the terms & conditions are applied in B2B-cases. This case is pretty obvious: you got what you signed up for. CrowdStrike with full scale access to your machines and no guarantees. On the other hand, Crowdstrike lost 125 billion in market cap. That's an indication of {liabilities + loss of future profits}. Pretty massive event for not being willing to do staging. But I expect it's mostly that CrowdStrike is tainted from now on. A friend of mine had a very bad stint as an employee of CrowdStrike recently and from what I learned from that case, I'm happy that the nature of the firm is somewhat more in the open now.
by jeffrallen on 7/25/24, 12:32 PM
That would have been literally the headline I'd choose for the bug.
This is incompetence that in a just world would result in the corporate death penalty.
by Baguette5242 on 7/25/24, 10:43 AM
- Is it reasonable to grant such privilege access to a piece of software that ultimately is a black box ?
- Is it reasonable to put a Microsoft / Commercial / Closed source OS in critical infrastructure ? If not considered as critical, then “important” infrastructure ?
- Is it reasonable to have more than 70% of the computers/servers that run important infrastructure on the same OS / software ? How about the mitigation of the risks etc…
I sincerely hope that all of this CrowdStrike mayhem will push stakeholders to draw some conclusions and actions.
by praptak on 7/25/24, 10:52 AM
by webworker on 7/26/24, 4:48 AM
by r00f on 7/25/24, 11:50 AM
by udev4096 on 7/25/24, 10:46 AM
by ChrisArchitect on 7/25/24, 12:06 PM
by threesevenths on 7/25/24, 12:17 PM
by pm2222 on 7/25/24, 12:27 PM
by justinclift on 7/25/24, 10:38 AM
Maybe now ClownStrike will start testing it properly, hopefully thereby fixing the stability and other issues.
by HenryBemis on 7/25/24, 11:28 AM
To whoever does this I have only one quote from Jaws:
You go in the cage, cage goes in the water, you go in the water, shark's in the water, our shark. Farewell and adieu to you, fair Spanish ladies. Farewell and adieu, you ladies of Spain.
by kierenj on 7/25/24, 10:56 AM
Eh, parts of this article aren't very reasonable. Even if they did a buttload of testing, it only takes one failure in one part of the chain (near the end).
They didn't test something they should have, sure, but obviously they didn't do "no testing whatsoever"
by elAhmo on 7/25/24, 10:57 AM
by hggh on 7/25/24, 12:04 PM
...based on the OVH precedent
by null_investor on 7/25/24, 11:37 AM
Corporativism in US is a thing. Companies can brick hospital systems killing patients, drive self-driving cars and run over people but don't get sued, and if they do, they settle for very little.
Just look at the recent Boeing incident where people were killed, the company clearly misled the US authorities and settled only a $0.5B fine.
Those companies in those scenarios should pay the fine that they should ($20B+), and if it means the company would go bankrupt, do it and form a new company diluting the previous shareholders.
Without doing this, shareholders and CEOs will have the incentive to carry on with their unfair practices that leads to dead people and deadlocked systems.