from Hacker News

TDD from the Factorio Team

by sorahn on 6/18/21, 10:53 AM with 303 comments

• by dgb23 on 6/18/21, 11:32 AM

  Two interesting takeaways:

  > This is the beautiful thing about having a company that isn't on the stock market. Imagine you have a company that goes slower and slower every quarter, and then you confront the shareholders with the statement, that the way to solve it, is to do absolutely no new features for a quarter or two, refactor the code, learn new methodologies etc. I doubt that the shareholders would allow that. Luckily, we don't have any shareholders, and we understand the vital importance of this investment in the long run. Not only in the project, but also in our skill and knowledge, so we do better next time.

  This is reassuring the notion of what I think actually matters, what the real essence is of developing a product, may that be a piece of art and entertainment (like here) or a productivity tool etc.

  There are creators and there are consumers. We split them up by developers, designers, domain experts and so on, but what matters is that all the other participants, especially those who can exert power traditionally are not part of the essence and if not being careful and responsible, can easily add complexity and limitations that are entirely accidental and can even be harmful.

  This reminds me of the agile manifesto, modern UX approaches and other processes that are driven by creators, but are often and very unfortunately being bent over backwards to fit into hierarchical power structures.

  > TDD actually is the constant fast swithing between extending the tests and making them pass continously. So as you write tests, you write code to satisfy them basically at the same time. This allows you to instantly test what you write, and mainly use tests as specifiation of what the code should acctually do, which guides the thought process to make you think about where you are headed to, and to write code that is more structured and testable from the very beginning.

  The important notion here is that TDD is not about tests and correctness, but about development. It continuously checks assumptions and explores the surrounding code, state and data until a sufficient solution is found.

  If we squint a little we can see how closely related TDD with REPL Driven Development is. In essence it is the same thing and even has similar results, where the tests or REPL code can be left as an artifact for further, likely historical understanding.

  We know now that neither is sufficient for a high degree of correctness, but they are certainly useful for understanding and development.

• by ramblerman on 6/18/21, 11:56 AM

  I'd be curious to hear what kovarex thinks in 2-3 months.

  TDD is often sold as a fix-all solution, which is incredibly appealing to mgmt and quite fun for most programmers as a new paradigm, allowing for quick adoption.

  It also has its uses, especially in the enterprise space where requirements aren't often clear. But I don't know many good programmers that truly stick to the dogma after the honeymoon period. It becomes just another tool in your toolset.

  Uncle bob is a salesman, not a "craftsman".

• by ineedasername on 6/18/21, 7:21 PM

  Every time a Factorio thread makes it to HN I feel like a fully recovered meth addict who suddenly has their old dealer knocking at their door:

  --

  Dealer: "Hey? You there? I got some meth for you."

  Me: "Go away! I don't want any!"

  Dealer: "Oh now don't say that. You remember how good it is? I know you do."

  Me: "I can't, I can't afford it, the price is too high."

  Dealer: "What? Come on, it's free! You already paid for it!.

  Me: "I'll lose my job, I can't, just go away!"

  Dealer: "Your JOB? This IS your job. Open the damn door, THE FACTORY MUST GROW"

  Me: ::cowers in the closet chanting please leave please leave please leave::

  --

  Also it was never good. It was more like a mind virus. Like the sort of problem or project at work you can't stop thinking about until it's done. Only with Factorio, it's never done. Never.

  My best defense against it are other video games I can stop & start when needed. Or booting up my VPN connection and picking a work task from my back log until the cravings go away.

• by truncate on 6/18/21, 11:41 AM

  > (1) no new features for a quarter or two, refactor the code, learn new methodologies etc

  > (2) This allows you to instantly test what you write, and mainly use tests as specification

  > (3) the problem comes when you break something and a lot of tests start to fail suddenly

  My favorites. Don't expect to give away entire quarter, but at-least sometime would definitely be nice. All three so fundamental, and often ignored. In my experience, you get these right, it makes developer life so much easier. As someone earlier mentioned in thread, TDD is kind of like REPL driven development.

  I think, one immediate benefit of companies focusing on good code is that engineers can aim for much more ambitious projects, and they can be more brave with the codebase. Instead we often end up with 100 over-engineered components with no well defined/enforced contracts, and a set of monolithic tests which runs the entire stack to test the most basic case.

• by blindmute on 6/18/21, 3:19 PM

  I'm not sure I understand why they're committing to such a long term refactor for a game which has already reached the tail end of its sales curve. As far as I know there are no internal monetization schemes in Factorio, and I really doubt further updates will boost sales anywhere near enough to justify the dev salaries.

• by IMTDb on 6/18/21, 11:54 AM

  > Imagine you have a company that goes slower and slower every quarter, and then you confront the shareholders with the statement, that the way to solve it, is to do absolutely no new features for a quarter or two, refactor the code, learn new methodologies etc. I doubt that the shareholders would allow that

  > now there are 9 programmers

  Companies on the stock market don't have "9 programmers". They have a lot of teams of 9 programmers. So while it's true that it would probably completely be impossible for a stock market company to completely freeze for a quarter or two, individual teams can still do that.

  If the factorio team grows to tens of programmers (it probably won't and probably shouldn't), I would be very surprised if they find the need - and if they manage to - freeze all teams together for a big refactoring round. I am also unsure that it would be the right approach. That observation holds wether they go public or stay private.

• by tobyhinloopen on 6/18/21, 11:32 AM

  I wonder how common TDD is in game development land, especially when you’re using things like Unity or Unreal.

  I feel like testing your behaviors is pretty hard, and even if you unit-test your behaviors, there’s still integration tests.

  I only write games as a hobby and never use TDD, even if I’d like to, since the tooling is just either poorly documented or too slow, or both.

  Usually this ends with me being frustrated with the slow development cycle and pushes me towards more unconventional methods of developing games in Javascript using Mocha to run the tests directly in the browser.

• by kevmo314 on 6/18/21, 11:10 AM

  > Which is a big improvement already, as adding and maintaining the new logic only requires you to look at one place instead of several, and it makes it generally more readable and less prone to errors.

  It's interesting to think about the other HN thread discussion about comments vs one-time-call function abstractions in this light: https://news.ycombinator.com/item?id=27546135

  I'm a big fan of "put code in one place" too. It was the biggest factor that convinced me that JSX was a great idea compared to separating the templating logic out.

• by ashtonkem on 6/18/21, 12:19 PM

  Honestly, I think the factorio team probably now knows more than Uncle Bob does, based on their blog posts.

• by nanis on 6/18/21, 2:50 PM

  This is a neat article. I do have comments about testing in general though.

  IME most developer do not understand each test has four possible outcomes:

  * Code is good and test passes

  * Code is bad and test fails

  These are the only two possible outcomes developers focus on: When I ask what they should do if a test that used to pass now fails, they always tell me stories about how to debug the code under test.

  There are two additional possibilities in test:

  * Code is bad yet test passes (false negative)

  * Code is good yet test fails (false positive)

  Again, IME, most people do not look at the test again once it passes for the first time.

  As a result, tests which are themselves code, become the largest untested part of the code base. You get these thousands and thousands of lines of untested code yet you have 100% code coverage.

  Some of my blog posts on testing:

  * Deception in tests considered harmful https://www.nu42.com/2017/02/deception-in-tests-harmful.html

  * Know what you are testing: The case of the test for median in Boost.Accumulators C++ Library <https://www.nu42.com/2016/12/cpp-boost-median-test.html>

  * Who is testing the tests? https://www.nu42.com/2015/05/who-is-testing-the-tests.html

  * Slashing one's feet with tests, or, how to fix 2,950 test failures in one fell swoop https://www.nu42.com/2015/08/fix-2950-test-failures.html

• by harryf on 6/18/21, 11:33 AM

  Came here hoping they’d turned Factorio into a tool of creating tests in other codebaes. Like literal gamification of work.

• by tgtweak on 6/18/21, 2:25 PM

  Damn I misread TTD and got excited that they were building it into factorio... Great article though, was not dissapointed.

• by achairapart on 6/18/21, 11:11 AM

  Warning: This page almost crashed my browser (FireFox on MacOS) and put my CPU on fire.

• by bluGill on 6/18/21, 3:22 PM

  > Imagine you have a company that goes slower and slower every quarter, and then you confront the shareholders with the statement, that the way to solve it, is to do absolutely no new features for a quarter or two, refactor the code, learn new methodologies etc. I doubt that the shareholders would allow that.

  It is called restructuring and big companies do it all the time. Investors allow it, though they are rightly suspicious - sometimes it is good, but often it is change for the sake of change and not change for better.

• by happyweasel on 6/18/21, 12:05 PM

  You can TDD as much as you want to once the initial game mechanics are in place and a gameprotoype shows enough promise to be realized until completion. Because then most of the core stuff/ideas/principles won't wildly change and won't be thrown away.. The core mechanics are in place. But would TDD help you reach that stage? I guess it is simply too much overhead. So yeah, this is TDD after the fact ;-). I still love TDD :)

• by Aardwolf on 6/18/21, 12:53 PM

  I used to follow friday facts until it stopped being weekly. I'm glad that every friday fact now gets posted on hacker news, that serves as my notification for new ones :)

  I also misread TDD as TTD (related to the trains in factorio) first

• by AwaAwa on 6/19/21, 12:04 PM

  While I'm ambivalent on TDD, there seems to be an attempt at cancellation brewing for his invocation of Uncle Bob.

• by swiley on 6/18/21, 2:31 PM

  Factorio convinced me that some people still write good closed commercial games. I wish the best for the authors and hope they don't stop any time soon.