by chaosprint on 5/9/25, 9:11 AM with 570 comments
by kion on 5/9/25, 9:04 PM
This is partly due to how we've distributed software over the last 40 years. In the 80s the idea of a library of functionality was something you paid for, and painstakingly included parts of into your size constrained environment (fit it on a floppy). You probably picked apart that library and pulled the bits you needed, integrating them into your builds to be as small as possible.
Today we pile libraries on top of libraries on top of libraries. Its super easy to say `import foolib`, then call `foolib.do_thing()` and just start running. Who knows or cares what all 'foolib' contains.
At each level a caller might need 5% of the functionality of any given dependency. The deeper the dependency tree gets the more waste piles on. Eventually you end up in a world where your simple binary is 500 MiB of code you never actually call, but all you did was take that one dependency to format a number.
In some cases the languages make this worse. Go and Rust, for example, encourage everything for a single package/mod to go in the same file. Adding optional functionality can get ugly when it would require creating new modules, but if you only want to use a tiny part of the module, what do you do?
The only real solution I can think of to deal with this long term is ultra-fine-grained symbols and dependencies. Every function, type, and other top-level language construct needs to declare the set of things it needs to run (other functions, symbols, types, etc). When you depend on that one symbol it can construct, on demand, the exact graph of symbols it needs and dump the rest for any given library. You end up with the minimal set of code for the functionality you need.
Its a terrible idea and I'd hate it, but how else do you address the current setup of effectively building the whole universe of code branching from your dependencies and then dragging it around like a boat anchor of dead code.
by socalgal2 on 5/9/25, 11:57 PM
Also there's arguably design. Should a 'glob' library actually read the file system and give you filenames or should it just tell you if a string matches a glob and leave the reset to you? I think it's better design to do the later, the simplest thing. This means less dependencies and more flexibility. I don't have to hack it or add option to use my own file system (like for testing). I can use it with a change monitoring system, etc...
And, I'm sure there are tons of devs that like the glob is a "Do everything for me" library instead of a "do one specific thing" library which makes it worse because you get more "internet points" the more your library doesn't require the person using it to be a good dev.
I can't imagine it's any different in rust land, except maybe for the executable thing. There's just too many devs and all of them, including myself, don't always make the best choices.
by jerf on 5/9/25, 1:58 PM
If I were designing a new language I think I'd be very interested in putting some sort of capability system in so I can confine entire library trees safely, and libraries can volunteer somehow what capabilities they need/offer. I think it would need to be a new language if for no other reason than ecosystems will need to be written with the concept in them from the beginning.
For instance, consider an "image loading library". In most modern languages such libraries almost invariably support loading images from a file, directly, for convenience if nothing else. In a language that supported this concept of capabilities it would be necessary to support loading them from a stream, so either the image library would need you to supply it a stream unconditionally, or if the capability support is more rich, you could say "I don't want you to be able to load files" in your manifest or something and the compiler would block the "LoadFromFile(filename)" function at compile time. Multiply that out over an entire ecosystem and I think this would be hard to retrofit. It's hugely backwards incompatible if it is done correctly, it would be a de facto fork of the entire ecosystem.
I honestly don't see any other solution to this in the long term, except to create a world where the vast majority of libraries become untargetable in supply chain attacks because they can't open sockets or read files and are thus useless to attackers, and we can reduce our attack surface to just the libraries that truly need the deep access. And I think if a language came out with this design, you'd be surprised at how few things need the dangerous permissions.
Even a culture of minimizing dependencies is just delaying the inevitable. We've been seeing Go packages getting supply-chain-attacked and it getting into people's real code bases, and that community is about as hostile to large dependency trees as any can be and still function. It's not good enough.
by zaptheimpaler on 5/9/25, 9:36 PM
In the absence of a technical solution, all others basically involve someone else having to audit and constantly maintain all that code and social/legal systems of trust. If it was pulled into Rust stdlib, that team would be stuck handling it, and making changes to any of that code becomes more difficult.
by palata on 5/9/25, 9:34 AM
Cargo makes it so simple to add tons of dependencies that it is really hard not to do it. But that does not stop here: even if I try to be careful with adding dependencies, a couple dependencies are likely to pull tens of transitive dependencies each.
"Then don't depend on them", you say. Sure, but that means I won't write my project, because I won't write those things from scratch. I could probably audit the dependency (if it wasn't pulling 50 packages itself), but I can't reasonably write it myself.
It is different with C++: I can often find dependencies that don't pull tens of transitive dependencies in C++. Maybe because it's harder to add dependencies, maybe because the ecosystem is more mature, I don't know.
But it feels like the philosophy in Rust is to pull many small packages, so it doesn't seem like it will change. And that's a pity, because I like Rust-the-language better than C++-the-language. It just feels like I trade "it's not memory-safe" for "you have to pull tons of random code from the Internet".
by Orangeair on 5/9/25, 9:02 PM
by nemothekid on 5/9/25, 2:08 PM
tokio is a work-stealing, asynchronous runtime. This is a feature that would be an entire language. Does OP consider it reasonable to audit the entire Go language? or the V8 engine for Node? v8 is ~10x more lines than tokio.
If Cloudflare uses Node, would you expect Cloudflare to audit v8 quarterly?
by neilv on 5/9/25, 1:52 PM
But that's not practical for all situations. For example, Web frontend developer culture might be the worst environment, to the point you often can't get many things done in feasible time, if you don't adopt the same reckless practices.
I'm also seeing it now with the cargo-culting of opaque self-hosted AI tools and models. For learning and experimenting, I'd spend more time sufficiently compartmentalizing an individual tool than with using it.
This weekend, I'm dusting off my Rust skills, for a small open source employability project (so I can't invest in expensive dependency management on this one). The main thing thing bothering me isn't allocation management, but the sinking feeling when I watch the cast-of-thousands explosion of transitive dependencies for the UI and async libraries that I want to use. It's only a matter of time before one of those is compromised, if not already, and one is all it takes.
by schmichael on 5/9/25, 2:06 PM
1. Well defined scope
2. Infrequent changes
Nomad has many of these (msgpack, envparse, cli, etc). These dependencies go years without changing so the dependency management burden rapidly approaches zero. This is an especially useful property for “leaf” dependencies with no dependencies of their own.
I wish libraries could advertise their intent to be Mature. I’d choose a Mature protobuf library over one that constantly tweaked its ergonomics and performance. Continual iterative improvement is often a boon, but sometimes it’s not worth the cost.
by Charon77 on 5/10/25, 2:03 AM
cargo tree helps a lot on viewing dependency tree. I forgot if it does LoC count or not..
> to see what lines ACTUALLY get compiled into the final binary,
This doesn't really make much sense as a lot of the functions that make it to the binary get inlined so much that it often becomes part of 'main' function
by rs186 on 5/9/25, 8:46 PM
Meanwhile, the heaviest JavaScript parser implemented in JavaScript is more lightweight.
I decided that I should leave this project alone and spend my time elsewhere.
by XxiXx on 5/9/25, 1:42 PM
by the__alchemist on 5/9/25, 11:20 PM
Curate a collection of libraries you use and trust. This will probably involve making a number of your own. Wheel-reinvention, if you will. If done properly, even the upfront time cost will save in the long-run. I am in the minority here, but I roll my own libs whenever possible, and the 3rd party libs I use are often ones I know, have used been for, and vetted that they have a shallow tree of their own.
Is this sustainable? I don't know. But It's the best I've come up with, in order to use what I see as the best programming language available for several domains.
There are a lot of light-weight, excellent libs I will use without hesitation, and have wide suitability. Examples:
- num_enum
- num_traits
- bytemuck
- chrono
- rand
- regex
- bincode
- rayon
- cudarc
- EGUI
- WGPU
- Winit
by klooney on 5/9/25, 1:28 PM
How much maintenance could you possibly need to load secrets from .env into the environment.
by QuadmasterXLII on 5/9/25, 10:06 PM
by sitta on 5/10/25, 1:01 AM
by cryptonector on 5/10/25, 6:42 AM
Big things you use off-the-shelf libraries for. Small things you open-code, possibly by cribbing from suitably-licensed open source libraries. You bloat your code to some degree, but reduce your need to audit external code and reduce your exposure to supply chain attacks. Still, the big libraries are a problem, but you're not going to open code everything.
This isn't just Rust. It's everything.
by colanderman on 5/9/25, 10:37 PM
by wpollock on 5/10/25, 12:15 AM
The tools I have found useful are:
cargo outdated # check for newer versions of deps
cargo deny check # check dependency licenses
cargo about # generate list of used licenses
cargo audit # check dependencies for known security issues
cargo geiger # check deps for unsafe rust
I haven't found a cargo tool I like for generating SBOMs, so I installed syft and run that.
cargo install-update # keep these tools updated
cargo mutants # not related to deps, but worth a mention, used when testing.
Having configured all these tools once and simply unzipping a template works well for me.
Suggestions for different or additional tools welcome!
Disclaimer: I'm not a professional rust developer.
by mleonhard on 5/9/25, 11:58 PM
I did this and it only solved half of the bloat:
https://crates.io/crates/safina - Safe async runtime, 6k lines
https://crates.io/crates/servlin - Modular HTTP server library, threaded handlers and async performance, 8k lines.
I use safina+servlin and 1,000 lines of Rust to run https://www.applin.dev, on a cheap VM. It serves some static files, a simple form, receives Stripe webooks, and talks to Postgres and Postmark. It depends on some heavy crate trees: async-fs, async-net, chrono, diesel, rand (libc), serde_json, ureq, and url.
2,088,283 lines of Rust are downloaded by `cargo vendor` run in the project dir.
986,513 lines using https://github.com/coreos/cargo-vendor-filterer to try to download only Linux deps with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu`. This still downloads the `winapi` crate and other Windows crates, but they contain only 22k lines.
976,338 lines omitting development dependencies with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu --keep-dep-kinds=normal`.
754,368 lines excluding tests with `cargo vendor-filterer --platform=aarch64-apple-darwin --exclude-crate-path='*#tests' deps.filtered`.
750k lines is a lot to support a 1k-line project. I guess I could remove the heavy deps with another 200 hours of work, and might end up with some lean crates. I've been waiting for someone to write a good threaded Rust Postgres client.
by aliceryhl on 5/9/25, 2:04 PM
by wyldfire on 5/10/25, 2:46 AM
Out of those 3.6 million lines, how many are lines of test code?
by gxt on 5/9/25, 2:22 PM
You can ensure that third-party Rust dependencies have been audited by a trusted entity with cargo-vet.
And you should have taken a look at where those 3M locs come from, it's usually from Microsoft's windows-rs crates that are transitively included in your dependencies through default features and build targets of crates built to run on windows.
by endorphine on 5/10/25, 4:15 AM
by GuB-42 on 5/10/25, 2:37 PM
I believe that it causes more problems than it solves, but it can be a solution to the problem of adding thousands of lines of code of dependency when you could write a 10-line function yourself.
Of course, the proper thing to do is not to be the wrong kind of lazy and to understand what you are doing. I say the wrong kind of lazy because there is a right kind of lazy, and it is about not doing things you don't need to, as opposed to doing them poorly.
by Avi-D-coder on 5/10/25, 4:42 AM
The author is right there's no way an individual can audit all that code. Currently all that code can run arbitrary build code at compile time on the devs machine, it can also run arbitrary unsafe code at runtime, make system calls, etc..
Software is not getting simpler, the abundance of high quality libraries is great for Rust, but there are bound to be supply chain attacks.
AI and cooperative auditing can help, but ultimately the compiler must provide more guarantees. A future addition of Rust should come with an inescapable effect system. Work on effects in Rust has already started, I am not sure if security is a goal, but it needs to be.
by demarq on 5/9/25, 12:13 PM
This is the way.
by thrance on 5/9/25, 1:53 PM
If I was to design a Rust 2.0, I'd make it so dependencies need permissions to access IO, or unsafe code, etc.
by James_K on 5/9/25, 11:13 PM
by bsrkf on 5/9/25, 11:40 PM
a) Ginger Bill (the Odin language creator, no affiliation) stated on a podcast that Odin will never have an official pkg manager, since what they're, in his opinion, mainly automating is dependency hell, and this being one of the main reasons for rising software complexity and lower software quality; see https://www.youtube.com/watch?v=fYUruq352yE&t=11m26s (timestamped to the correct position) (they mention Rust explicitly as an example)
b) another programmer rather seriously worried about software quality/complexity is Jonathan Blow, who's talk "Preventing the Collapse of Civilization" is worth watching in my opinion: https://www.youtube.com/watch?v=ZSRHeXYDLko (it's not talking about package managers specifically, but is on topic regarding software complexity/quality as a whole)
Addendum: And sorry, I feel like almost everyone knows this xkcd by now, but since no one so far seems to have posted it; "obligatory xkcd reference": https://imgs.xkcd.com/comics/dependency_2x.png
by stefanos82 on 5/9/25, 9:36 AM
by weltensturm on 5/10/25, 11:11 AM
The solution space is basically infinite, and that's a good thing for a systems programming language. It's kind of amazing how far rust reaches into higher level stuff, and I think the way too easy to use package manager and lively crate ecosystem is a big part of that.
Sometimes I wish for a higher-level rust-like language though, opinionated as hell with garbage collector, generic functions without having to specify traits, and D's introspection.
by iainmerrick on 5/10/25, 3:44 PM
The total number of lines of code is relevant, sure, but for most practical purposes, compile times and binary sizes are more important.
I don't know the situation in Rust, but in JS land, there's a pretty clear divide between libraries that are tree-shakable (or if you prefer, amenable to dead code elimination) and those that aren't. If you stick to tree-shakable dependencies your final bundled output will only include what you actually need and can be pretty small.
by yubblegum on 5/10/25, 5:53 PM
https://docs.osgi.org/specification/osgi.core/7.0.0/framewor...
"How OSGi Changed My Life" (2008) https://queue.acm.org/detail.cfm?id=1348594
by vladkens on 5/10/25, 1:00 AM
by vsgherzi on 5/9/25, 11:03 PM
by CraigJPerry on 5/10/25, 8:40 AM
But you’re still open to typo squatting and similar issues like crates falling unmaintained - the article mentions the now famous dotenv vs. dotenvy issue (is this solvable with a more mature governance model for the crates ecosystem? At this point dotenv should probably be reclaimed). So after vendoring a baseline set of dependencies, you need to perform comprehensive auditing.
Maybe you can leverage LLMs to make that blob of vendored deps smaller / cheaper to own. Maybe you can distill out only the functionality you need (but at what cost, now you might struggle to backport fixes published upstream). Maybe LLMs can help with the auditing process itself.
You need a stream of notifications of upstream fixes to those vendored deps. Unfortunately in the real world the decision making will be harder than “ooh, there’s a sec fix, I should apply that”.
I always wonder why someone like JFrog don’t expand their offering to provide “trusted dependencies” or something similar. I.e. you pay to outsource that dependency governance and auditing. Xray scanning in the current product is a baby step toward the comprehensiveness I’m suggesting.
Taking a step back though, I’d be really careful not to throw the baby out with the bath water here. Rust has a fairly unique capability to compose work product from across unrelated developers thanks to its type system implementation (think about what happens with a C library, who’s responsible for freeing the memory, you or me?). Composition at scale is rusts super power, at least in terms of the productivity equation for large enterprises - in this context memory safety is not the sales pitch since they already have Java or whatever.
by morsecodist on 5/10/25, 11:44 AM
by vzaliva on 5/9/25, 11:49 PM
by righthand on 5/9/25, 2:14 PM
by nyuriks on 5/10/25, 6:14 AM
by rvz on 5/10/25, 1:42 AM
A proper 'batteries included' standard library in the language and discouraging using too many libraries in a project.
The same mistakes from the Javascript community are being repeated in front of us for Cargo (and any other project that uses too many libraries).
by NewJazz on 5/10/25, 4:15 AM
Tokei hasn't had a stable release in over 4 years and misreports lines of code in some instances. The author in the past has basically said they would need to be paid to backport one line fixes with no merge conflicts that fix real accuracy issues in their software... Bad look in my book.
by arp242 on 5/10/25, 4:32 AM
I'm not very familiar with Rust, but all of Go is 1.6M lines of Go code. This includes the compiler, stdlib, tests for it all: the lot.
Not that I doubt the sincerity of the author of course, but maybe some irrelevant things are counted? Or things are counted more than once? Or the download tool does the wrong thing? Or there's tons of generated code (syscalls?)? Or ... something? I just find it hard to believe that some dependencies for web stuff in Rust is twice all of Go.
by ak_111 on 5/10/25, 9:33 AM
I am wondering if there is a good modern reference that provides a conceptual overview or comparative study of the various techniques that have been attempted.
It is a hard subject to define as it cuts through several layers of the stack (all the way down to the compiler system interface layer), and most book focus on one language or build technology rather than providing a more conceptual treatment of the techniques used.
by kirici on 5/10/25, 7:57 AM
"A little copying is better than a little dependency." - grab the parts that you need and then include the library only in a test to ensure alignment down the line, an idea I liked a lot.
by amelius on 5/9/25, 10:20 PM
by conradludgate on 5/9/25, 9:27 PM
If you want to use dependencies, I wouldn't be surprised when you realise they also want to use dependencies. But you can put your money/time in the right places. Invest in the dependencies that do things well.
by philsnow on 5/9/25, 10:59 PM
I'm not saying you copy-pasted those 35 lines from dotenvy, but for the sake of argument let's say you did: now you can't automatically benefit from dotenvy patching some security issue in those lines.
by lmm on 5/10/25, 2:43 AM
by anothernewdude on 5/10/25, 11:34 AM
by bradley13 on 5/10/25, 12:16 PM
There's no good solution...
by hedora on 5/10/25, 3:00 AM
It lets security professionals cryptographically vouch for the trustworthiness of rust packages.
by ThouYS on 5/10/25, 3:28 PM
by hedora on 5/10/25, 1:59 AM
It lets security professionals audit rust packages, and cryptographically attest to their trustworthiness.
by swoorup on 5/10/25, 9:27 AM
by viktorcode on 5/10/25, 11:05 AM
by srikanth767 on 5/9/25, 1:58 PM
by 1vuio0pswjnm7 on 5/9/25, 3:25 PM
Isn't the point of a memory safe language to allow programmers to be sloppy without repercussions, i.e., to not think about managing memory and even to not understand how memory works.
Would managing dependencies be any different. Does Rust allow programmers to avoid thinking carefully about selecting dependencies.
by csomar on 5/9/25, 1:56 PM
This is a problem with all languages and actually an area where Rust shines (due to editions). Your pulled in packages will compile as they previously did. This is not true for garbage collected languages (pun intended).
> Out of curiosity I ran toeki a tool for counting lines of code, and found a staggering 3.6 million lines of rust .... How could I ever audit all of that code?
Again, another area where Rust shines. You can audit and most importantly modify the code. This is not that easy if you were using Nodejs where the runtimes are behind node/v8 or whatever. You compile these things (including TLS) yourself and have full control over them. That's why Tokio is huge.
