by fagnerbrack on 12/22/24, 3:12 AM with 93 comments
by sourceless on 12/22/24, 8:35 AM
The author notes that there appears to be a fixed number of changes per deployment and that it is hard to increase - I think the 'Reversie Thinkie' here (as the author puts it) is actually to decrease the number of changes per deployment.
The reason those meetings exist is because of risk! The more changes in a deployment, the higher the risk that one of them is going to introduce a bug or operational issue. By deploying small changes often, you get deliver value much sooner and fail smaller.
Combine this with techniques such as canarying and gradual rollout, and you enter a world where deployments are no longer flipping a switch and either breaking or not breaking - you get to turn outages into degradations.
This approach is corroborated by the DORA research[0], and covered well in Accelerate[1]. It also features centrally in The Phoenix Project[2] and its spiritual ancestor, The Goal[3].
[1] https://www.amazon.co.uk/Accelerate-Software-Performing-Tech...
[2] https://www.amazon.co.uk/Phoenix-Project-Helping-Business-An...
[3] https://www.amazon.co.uk/Goal-Process-Ongoing-Improvement/dp...
by lifeisstillgood on 12/22/24, 3:51 PM
This leads to a few corollaries - things like “If GPUs do the work then coders are the new managers” or we need whole-org-test-rigs to be clear about the impacts of chnages.
This seems directly related to this excellent article - to my mind if all the decision makers are not looking at the code as the first class object in a chnage process (is opposed to Jiras or project plans) then not all decision makers are (software) literate - and this comes up a lot in the threads here (“how do I discuss with non-technical management”) - the answer is you cannot - that management must be changed. This is an enormous generational road block that I thought was a problem thirty years ago but naively assumed would disappear as coders grew up. Of course the problem is that to “run” a company one does not need to code - so until not coding is something embarrassing like not writing is for a newspaper editor we won’t get past it.
The main point is that we need companies that can be run with the new set of self-reinforcing concepts - sops, testing, not meetings but systems as communication.
I will try and rewrite this comment later - it needs work
by andy_ppp on 12/22/24, 11:15 AM
In the end fighting the bureaucracy is actually impossible in most organisations, especially if you’re not part of the 200 layers of management that create these meetings. I would sack everyone but programmers and maybe two designers and let everyone fight it out without any agile coaches and product owners and scrum master and product experts.
Slow deployment is a problem but it’s not the problem.
by braza on 12/22/24, 5:34 PM
When something happens in production we ended up placing more tests; and of course when things goes south at least 50 minutes were necessary to recover.
After a lot of consideration we decided to focus on the recovery and relax and simply some tests and focus on recovery (i.e. have the full thing in less than 5 minutes) combined with a canary as deployment strategy (instead rolling updates).
At least for us was a so refreshing experience but sounded wrong in some ways.
by qaq on 12/22/24, 5:41 AM
by vegetablepotpie on 12/22/24, 1:24 PM
I responded to this risk by making a meeting. I presented our project schedule, and told my colleagues about their expectations, I.e. if they drop code style comments on the PRs they will be deferred to a future PR (and then ignored and never done).
What we needed is fine grained testing with better isolation between components. The problem is is that our management is at a high level, they don’t see meetings as a means to an end, they see meetings as a worthy goal in and of itself self to achieve. More meetings means more collaboration, means good. I’d love to see advice on how to lead technical changes with non-technical management.
by dang on 12/22/24, 4:45 AM
Slow Deployment Causes Meetings - https://news.ycombinator.com/item?id=10622834 - Nov 2015 (26 comments)
by lizzas on 12/22/24, 5:12 AM
by austin-cheney on 12/22/24, 11:49 AM
TLDR; software performance, thus human performance, is all that matters.
Risk management/acceptance can be measured with numbers. In software this is actually far more straightforward than in many other careers, because software engineers can only accept risk within the restrictions of their known operating constraints and everything else is deferred.
If you want to go faster you need to maximize the frequency of human iteration above absolutely everything else. If a person cannot iterate, such as waiting on permissions, they are blocked. If they are waiting on a build or screen refresh they are slowed. This can also be measured with numbers.
If person A can iterate 100x faster than person B correctness becomes irrelevant. Person B must maximize upon correctness because they are slow. To be faster and more correct person A has extreme flexibility to learn, fail, and improve beyond what person B can deliver.
Part of iterating faster AND reducing risk is fast test automation. If person A can execute 90+% test coverage in time of 4 human iterations then that test automation is still 25x faster than one person B iteration with a 90+% lower risk of regression.
by jojobas on 12/22/24, 8:18 AM
by yarg on 12/22/24, 4:48 AM
by tpoacher on 12/22/24, 8:51 PM
But then there's Jira.
/s
by Sparkyte on 12/22/24, 5:03 AM