by mononcqc on 5/1/20, 11:31 AM with 171 comments
by JoeAltmaier on 5/1/20, 2:53 PM
So I took 2 days, made a chart of every path and pattern of events (restarting a timer from a timer callback; having another time interval expire while processing the previous; restarting while processing and another interval expires; restarted timer expires before completing previous expiration and on and on). Then writing exhaustive code to deal with every case. Then running every degenerate case in test code until it survived for hours.
It never had to be addressed again. But it did have to be addressed. So many folks are unwilling to face the music with complexity.
by UweSchmidt on 5/1/20, 5:27 PM
There are also ways to design things better (or worse) with regards how they handle complexity. Reuse UI, patterns, workflows and languages. Keep things consistent, make things discoverable.
Point is, the slogan "Complexity has to live somewhere" could also be used as an excuse to do sloppy work. Then again, this keeps us all employed.
by bertmuthalaly on 5/1/20, 2:40 PM
Otherwise this essay is spot on when it comes to essential conplexity.
Incidentally, the question of “where complexity lives” is one of the focal points of “A Philosophy of Software Design,” which comes highly recommended if you’re trying to come up with your strategy for managing complexity from first principles.
by spacedcowboy on 5/1/20, 2:46 PM
However, complexity can be managed. Humans do this using abstraction as a tool. We divide the complex problem up into a sequence of several simpler states, and we find it easier to understand the simpler problems along with the sequence within which they lie.
Good software uses this same approach to reduce complex issues to a manageable process. A good tool makes the simple things easy and the complex thing possible, the design of the tool reflects the effort and work that the designer out in to u the problem they’re trying to solve, and to produce something that helps guide others along that self-same path of understanding, without them having to put in the same level of effort; it establishes the golden path through the marshes and bogs of difficulties that the problem domain throws up.
“Embracing complexity” is a measure of last resort, IMHO. It means the tool developer could not analyze the problem and come up with a good solution; it means “here, you figure it out”; it means giving up on one of the fundamental reasons for the tools existence.
Sometimes, embracing complexity and the ensuing struggle that this necessitates is simply what you have to do, but not often. Maybe, maybe this is one of those times, but I always start off with a critical eye when someone tells me that a complicated thing is “the only way it can be done”. Colour me sceptical.
by deathanatos on 5/1/20, 4:48 PM
Essentially, the author is, I think, arguing about what the paper calls "Essential complexity", complexity inherent to the problem one is trying to solve. And with that, I agree.
I think the author should acknowledge accidental complexity (or provide some argument as to why that must live somewhere), and I think a lot of comments here on HN are pointing out the fact that accidental complexity exists, and doesn't have to live somewhere. But my guess is that that's not what the author is saying, and that the author is only arguing about essential complexity.
[1]: https://github.com/papers-we-love/papers-we-love/blob/master...
¹I personally found this paper somewhat mixed. The definitions of complexity are what make it worth reading. Its conclusion of "functional programming languages will fix all the woes" I think is not practical.
by x3haloed on 5/1/20, 11:17 PM
The failing of this point is that much of what we call complexity is disorganization. Certainly, there is a fundamental level of logic in any desired system that can not be willed away with cute patterns, but to consider all complexity of an existing system to be necessary is a fallacy. Dividing systems into problem domains does not inherently reduce the total complexity of a system. It probably usually adds to it. But organizing systems this way can drastically reduce the scope of complexity into manageable pieces so that mere mortals can work on it with out having to hold the entire system in their mind at one time.
It’s like saying that you can’t make a garage full of junk any less complicated, because no matter how you arrange it, it will still contain all the same junk. In fact, organizing all the junk into manageable storage can make it much easier to understand, work with, sort through, clean, and identify items that may be unnecessary.
by crazygringo on 5/1/20, 2:46 PM
Along the same lines, there's a great quote from many years ago that I unfortunately can't find the exact text of, but it goes like this (paraphrasing):
"Most Microsoft Word users only use 5% of its features."
"So why don't we get rid of the other 95%, since it's so bloated and complex?"
"Because each user uses a different 5%."
by marcosdumay on 5/1/20, 5:48 PM
So, no, it doesn't have to live somewhere. It can be created and destroyed, this happens every day. Probably some of it can not be destroyed, but nobody knows what part so any article about it will be useless.
by slx26 on 5/1/20, 7:11 PM
> We try to get rid of the complexity, control it, and seek simplicity.
Well, not really enough. Complexity is a beast that takes many years to understand, and that's when you are really trying. Many devs don't. And companies even less. So while it's true some complexity is unavoidable, I think we still have a long way to go in being aware of it first. We write software once, and that's not a proper strategy to manage complexity. Anyone who has looked into computers from top to bottom knows we have tons of problems with complexity that we really haven't solved properly, and that bleed into day to day development in the ugliest ways.
My particular take when I don't have the massive amount of time required to properly deal with complexity is write something like this: "hey, this is very complex. please don't touch it. if you have to, we have this much headroom. if you need to go beyond that, please rewrite this entirely / find a better way. if the code doesn't bite you immediately I will".
by 3pt14159 on 5/1/20, 2:52 PM
https://news.ycombinator.com/item?id=18774619
One of my highest upvoted comments with 161 upvotes.
But I've come to another idea too. Part of all this complexity is dealing with change. We could simplify things if some aspects of our software hit a final point where only security updates were published after that. Imagine a programming language that was specified and actually finished without the constant roll of changing patterns and practices. Or a web framework. Or a database. Intentionally designed to be robust and secure from the first day then intentionally set to minimal patches for bugs and security fixes.
Part of the issue though is that things keep changing. New characters are added to unicode, new timezones emerge or existing ones change. It's a hard problem to crack.
by apta on 5/1/20, 3:06 PM
* Any language has its paradigms that need to be learned, and golang is no different. Just because you can learn the syntax in a couple of sittings does not mean you know how to use the language in an effective matter. Something I see many people not bring up.
by ebiester on 5/1/20, 3:23 PM
If you are lucky, you have someone who is able to analyze the entire system, can identify all of the stakeholders, and can drive consensus when the change in abstractions is necessary, and has the budget to do it.
The danger is that people pick the solution for a large solution first because "we will need this someday" rather than waiting for the accidental complexity to build, knowing that the rework necessary to move to an intermediate system is less expensive than the cost of delay in getting the simple solution out first.
My dream is that we can build incremental complexity systems that could support simple solutions quickly and highly complex solutions eventually. The problem is that these are hard to build. :)
by aazaa on 5/1/20, 3:34 PM
I'm not sure I'd agree. Some complexity exists because the effort to simplify was too great. Either cost or skill prevented the refactor. As the saying goes, "sorry this letter is so long, I didn't have time to make it shorter."
I've repeatedly found that the first iteration of a solution tends to be more complex. One culprit is that sufficient abstractions were either not recognized or not implemented. Put those abstractions in, and you simplify the system.
So that refactoring step after an initial solution is created is crucial and unfortunately often just not done.
by JackRabbitSlim on 5/1/20, 3:41 PM
Now look at this app I can write in 30 lines of code with this framework! Oh and 500 lines of yaml across 2 dozen files but look the "code" is so sleek and sexy.
Joking aside I think he could have at least mentioned the difference between required complexity and unneeded complexity. Ironically it seems to spawn from attempts to reduce the first type of required complexity in a lot of cases.
by ken on 5/1/20, 5:03 PM
Dirt has to live somewhere, too. That's a good motto for a garbage company, but if you hear a chef saying it in the kitchen while preparing your meal, you might be worried.
by arendtio on 5/1/20, 10:07 PM
When I want to search for a string within some data, I don't want to have to think about the algorithm it uses (naive, Boyer–Moore [1], etc.). I just want to know that somebody cared about it and that it will work great.
Bad abstractions, on the other hand, make you think about the underlying details that you have to be aware of. For one layer that might not be too bad, but the more you build on top of such things, the more fragile the whole construct becomes.
[1] https://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_string-sea...
by peterwwillis on 5/1/20, 4:44 PM
Simplicity is the art of taking complex things and stripping them down to their essential nature. Simple things can be complex, but no more complex than they have to be.
With that said, this piece talks a lot about frameworks and tools and patterns, and I really hope people don't think that's the right way to think about architecture. Abstractions do not make things simpler, they make them abstract.
by zmmmmm on 5/1/20, 11:54 PM
by Discombulator on 5/2/20, 4:13 PM
I am fully on board with the overall sentiment of the article that there is some irreducible complexity that one cannot avoid. Often it comes straight from the business domain or entity the code is modelling, and then, sure, you cannot make it any easier, otherwise you are not solving the problem and users will be unhappy.
However, then the author goes too far: > Accidental complexity is just essential complexity that shows its age.
You can absolutely add heaps of unnecessary complexity; in fact, this is almost surely what you will get by attempting to cover every possible future use case and evolution scenario, or to cater to every minor aesthetic concern (e.g., "I need to be able to replace my cloud provider / DB / message queue / user notification medium / etc by changing just one line!").
It takes humility to admit that getting the balance "right" from the start is difficult, and often the only way to improve is to accept some badness now and revisit the decision later with more information.
The quote however seems to be saying that we (as software engineers) always get it right, just later things change and our choice does not appear right anymore. This mindset is counterproductive, as admitting flaws is the first step to improve.
by darkerside on 5/1/20, 5:26 PM
Doesn't that hold that the inverse must be true? In some cases, you must be able to remove complexity from the equation, without introducing it somewhere else.
I agree with the spirit of the post, but I think it's eliding some of the complexity involved in identifying essential complexity.
by BenoitEssiambre on 5/1/20, 9:14 PM
I tried to describe them here. I'm not sure I did a good job:
https://medium.com/@b.essiambre/product-market-crossfit-c09b...
The gist:
Low resolution fit: The product vaguely and simply fits its market or domain.
High resolution fit: The product is complex and is tightly tailored to the market or domain.
Overfit: The product is so tightly tailored to some users that it only fits a small part of the market.
Underfit: The product is so generic that it’s missing important features and corner cases.
And then another qualifier 'crossfit' which is a bit harder to grasp, inspired by cross-entropy, that has to do with whether the design language, which can be seen in some ways as 'encoding' the problem, fits the domain or market well.
by im3w1l on 5/1/20, 7:50 PM
by l0b0 on 5/1/20, 11:15 PM
by rhacker on 5/1/20, 6:38 PM
Yet for other people it's a simple as fuck thing that accomplishes things for them and lets them move forward with getting shit done.
So really, it's how you look at it.
The human body is so freaking complex, it is entirely possible we'll never even understand how half of it works. Yet from another point of view we easily hire a bunch of humans to do tasks (programming, making food, yard work, etc..). And even though we don't know how the body works, we have a simple understanding of how that body can accomplish those tasks, which is enough to move forward.
by senderista on 5/1/20, 6:51 PM
by mcqueenjordan on 5/1/20, 5:35 PM
Sometimes there is /essential complexity/, but it is far from always. I think the line of thinking espoused by the author leads to less simplification and encourages a "well, the complexity has to live somewhere, so why bother?" type of approach. However, reducing complexity is often possible, and it usually comes before the implementation stage. Reducing complexity sometimes means solving a different problem.
by throwaway55554 on 5/1/20, 6:05 PM
If you write anything, even "hello, world", complexity lives somewhere! Just the complexity of the print function path all the way to the hardware would baffle some.
by hasahmed on 5/1/20, 7:41 PM
by hzhou321 on 5/1/20, 3:57 PM
by smitty1e on 5/1/20, 3:15 PM
Keeping them orthogonal seems the first step in the process of trading the problem for a smaller problem.
As TFA notes, there is a lower limit to how much complexity can be squeezed out of a system.
by dasyatidprime on 5/1/20, 5:02 PM
by patkai on 5/1/20, 8:19 PM
by dana321 on 5/1/20, 8:27 PM
Go : Make the language simple, make achieving a simple thing complex.
C : The machine is complex, be careful what you do if you overflow into the inner workings.
by grensley on 5/1/20, 5:03 PM
by sumnole on 5/1/20, 6:42 PM
by philipswood on 5/2/20, 1:50 PM
by crimsonalucard on 5/1/20, 2:34 PM
The real question of design, especially the design and organization of computer programs, is where that complexity lives, and how do you organize that complexity so that it doesn't introduce extra complexity.
Usually the way we handle this is through layers, with the initial layers being simple, the middle layers being complex and the top layers being simple again. There's no theory about why this is better, it is just usually how it's done and it seems to work when we can pull it off.
The bottom layers are your primitives and axioms. Simple building blocks. This is where designers want to create a most minimal and simple set of tools that can help facilitate unlimited complexity at the upper layers (Think a minimal set of language primitives that allow for Turing completeness)
The middle layers are your theorems. Different permutations and compositions of your axioms to implement things like business logic or whatever logic you want. This is where good designers should try to stuff as much complexity into as possible.
The Top and final layer is the interface. Something that is built to directly access the middle layer while providing an interface that is simple and more understandable from the perspective of a user. This is probably the least theoretical aspect of the stack as you have to factor in things like art and human psychology into how you build an interface.
A good example of this will be your phone. The primitives are assembly language instructions, RISC. The middle layer is all the code that facilitates programming and applications and the interface layer is the touch screen.
Note that you don't have to look at it from a birds eye view from CPU instructions all the way to touch screen. You can zoom in and look at just a web stack: Java on the backend all the way up to react on the front end.
By zooming in you will see systems that violate the design principle I describe. Java as a primitive has become incredibly bloated and so has React and the javascript tool chain. There are several reasons for why this happens: Lack of planning, optimization requirements inevitably violate design principles, lack of knowledge, lack of central direction and more.
Either way, always remember that the ideal design that most people should strive for is complexity sandwiched by simplicity at the primitive layer and the interface layer. This goes for most things within computer programming and outside of it, like your car.
by carapace on 5/1/20, 5:43 PM