from Hacker News

Super Colliding Nix Stores: Nix Flakes for Millions of Developers

by osener on 5/25/23, 6:00 PM with 76 comments

• by peterldowns on 5/25/23, 7:54 PM

  I think the best part about Nix is that it makes it super easy to get software into your development environment. This layered store is a pretty cool setup to make that efficient for Replit.

  Not sure how if Nix/Replit has the program you’re looking for? Or what package name to use to actually get the program you want? Check out my cli search tool:

  https://github.com/peterldowns/nix-search-cli

  Edit: I can’t help myself from ranting, it’s insane that I had to write this tool. Speaks to the lack of user focus in the Nix community. Even the blog post here doesn’t mention how to actually find packages to install!

  What good is a package manager that makes finding packages difficult? Who should be expected to tolerate such a thing?

• by hbrundage on 5/26/23, 1:20 AM

  Dang this is cool! I get why replit went so heavy on nix but I also feel like it must have a cost for them — nix is hard to learn, especially for folks new to development which I know makes up a lot of replits customer base.

  We built a solution to the same problem with a similar approach[1], but that just snapshots any old files instead of doing nix derivations. Nix couples the build process to the content-addressability of the output, which works great if you want to put all the effort in to deterministic builds. We just read files like git does which works great for non-deterministic processes like npm install (tragically).

  I like the idea of the Big Disk style of attaching a content addressable cache, but in our experiments we still found the network latency to the attached disk too high when reading file by file, like when booting a node app, so we’re caching a much smaller amount on a local SSD for each prod server. Maybe replit isn’t as sensitive to read perf from the cache layer, or they have fancy local per-node read through caching within the overlay setup? Regardless, cool!!

  [1]: https://github.com/gadget-inc/dateilager

• by smasher164 on 5/25/23, 8:47 PM

  I love Nix as a package manager, and flakes are awesome for setting up environments. I wish it had more affordances for being a proper build system. If only there were a Buck2 and Nix hybrid.

• by tormeh on 5/26/23, 12:31 AM

  Nix Flakes are deemed experimental and explicitly not recommended in prod by the Nix project, afaict. Despite these warnings it’s widely used, but breakage is very much a possibility because it hasn’t gone through the Nix RFC process.

• by npub1zenn0 on 5/27/23, 11:44 AM

  I don't get how people find nix useful. Setting up a python environment, for example, is a total nightmare. Nix lang itself seems completely unnecessary. Guix is using scheme, which I assume people would be more incentivized to learn just because it's a general programming language. So why is nix winning?

• by Stammon on 5/25/23, 8:49 PM

  So do they share their implementation of overlaying stores somewhere? It would be a huge benefit for the community to have it open source and available for being used in countless other use cases.

• by nwmcsween on 5/26/23, 1:01 AM

  A less painful alternative to dancing around the issue is just to use the filesystem itself as a global database.

• by woleium on 5/25/23, 9:43 PM

  But oh my, the security implications. Nix is great, but there's no chain of trust on where these packages come from. Did we learn nothing from 'left_pad'?

• by firstlink on 5/26/23, 4:14 AM

  Nix is solving the wrong problems, and I'm dismayed that it is being adopted more and more to solve those wrong problems. The very concept of a "development environment" is indicative of broken build systems. Ecosystems like python are trying desperately to tame the insanity using venvs, while ecosystems like rust were built correctly from the start with Cargo. Neither of these benefits in the least way from Nix or the most popular alternative, Docker (except that both end up falling victim to the insanity of other ecosystems when FFI comes into play).

  What Nix could have been was a global un-fscked dependency manager, a la Portage but with Nix's special sauce for local environments -- thereby working around fundamentally and permanently broken ecosystems like C, which would be useful even in the FFI case. But somehow it never got around to actually managing dependencies. By failing to manage dependencies (by pretending that versioning doesn't exist, and supporting only exact software sets), Nix only re-entrenches the same bad behavior and broken systems which it should have been replacing: i.e., encouraging "works-on-my-system" build environments, by making them part of mandatory collaboration workflows.

  So, soon: Encounter a bug because the build depends on the environment in some inappropriate, insane way? Then it's your own fault for not adhering to the approved Nix build instructions which would set up a system which doesn't exhibit that bug.