by statenjason on 6/10/22, 7:35 PM with 186 comments
by bumper_crop on 6/10/22, 9:20 PM
I'll add one other data race goof: atomic.Value. Look at the implementation. Unlike pretty much every other language I've seen, atomic.Value isn't really atomic, since the concrete type can't ever change after being set. This stems from fact that interfaces are two words rather than one, and they can't be (hardware) atomically set. To fix it, Go just documents "hey, don't do that", and then panics if you do.
by smasher164 on 6/10/22, 11:59 PM
1. Closures and concurrency really don't mix well. The loop variable capture in particular is very pernicious. There's an open issue to change this behavior in the language: https://github.com/golang/go/issues/20733.
2. Yep. I've seen this problem in our codebase. I've grown to just be very deliberate with data that needs to be shared. Put it all in a struct that's passed around by its pointer.
3. This issue is caught fairly easily by the race detector. Using a sync.Map or a lock around a map is pretty easy to communicate with other Go devs.
4. This should be documented better, but the convention around structs that should not be passed around by value is to embed a noCopy field inside. https://github.com/golang/go/issues/8005#issuecomment-190753... This will get caught by go vet, since it'll treat it like a Locker.
5 & 6. Go makes it pretty easy to do ad-hoc concurrency as you see fit. This makes it possible for people to just create channels, waitgroups, and goroutines willy-nilly. It's really important to design upfront how you're gonna do an operation concurrently, especially because there aren't many guardrails. I'd suggest that many newcomers stick with x/sync.ErrGroup (which forces you to use its Go method, and can now set a cap on the # of goroutines), and use a *sync.Mutex inside a struct in 99% of cases.
7. Didn't encounter this that often, but sharing a bunch of state between (sub)tests should already be a red flag. Either there's something global that you initialized at the very beginning (like opening a connection), or that state should be scoped and passed down to that individual test, so it can't really infect everything around it.
by jchw on 6/10/22, 9:52 PM
I'm surprised by some of them. For example, go vet nominally catches misuses of mutexes, so it's surprising that even a few of those slipped through. I wonder if those situations are a bit more complicated than the example.
Obviously, the ideal outcome is that static analysis can help eliminate as many issues as possible, by restricting the language, discouraging bad patterns, or giving the programmer more tools to detect bugs. gVisor, for example, has a really interesting tool called checklocks:
https://github.com/google/gvisor/tree/master/tools/checklock...
While it definitely has some caveats, ideas like these should help Go programs achieve a greater degree of robustness. Obviously, this class of error would be effectively prevented by borrow checking, but I suppose if you want programming language tradeoffs more tilted towards robustness, Rust already has a lot of that covered.
by ohazi on 6/11/22, 4:23 AM
So now I'm hearing that Go, a garbage collected language, doesn't guarantee data race freedom? I guess it's garbage collected but not "managed" by a runtime or something?
Why go to all that effort to get off of C++ just to stop 30% short? These are C-like concurrency bugs, and you still have to use C-like "if not nil" error handling.
Why do people keep adopting this language? Where's the appeal?
by hintymad on 6/10/22, 11:47 PM
A side topic: this is really not something to be proud of. There used to be more people than quantity of work in Uber and engineers fought for credits by building bogus decomposed services, and the sheer number of services seems indicate it's still so.
by DominoTree on 6/11/22, 2:47 AM
"The key point here is our programmers... They’re not capable of understanding a brilliant language... So, the language that we give them has to be easy for them to understand"
by aaronbwebber on 6/10/22, 11:13 PM
https://go.dev/doc/articles/race_detector
Edit: at the _end_ of the post, they mention that this is the second of two blog posts talking about this, and in the first post they explain that they caught these by deploying the default race detector and why they haven't been running it as part of CI (tl;dr it's slower and more resource-expensive and they had a large backlog).
https://eng.uber.com/dynamic-data-race-detection-in-go-code/
by Klasiaster on 6/10/22, 11:22 PM
by dwrodri on 6/11/22, 7:36 PM
Here's a simple question that's stumped me for some time: if multiple go routines are popping values out of a channel, does the channel need a mutex? Why do the "fan-out, fan-in?" examples in the "Pipelines and Cancellation" post on the Go blog not require mutex locks? Link here: https://go.dev/blog/pipelines
Stuff like that, along with the ambiguity of intializing stuff by value vs using make, the memory semantics of some of the primitives (slices, channels, etc). None of it was like "of course". If something is a reference, I'd rather the language tell me it's a reference. Maybe I'm still too new to the language.
by eurasiantiger on 6/11/22, 6:53 AM
What the hell is that company doing?
Try to imagine an ERD or DFD of their day-to-day operations. 2,100 unique services…
by dubswithus on 6/10/22, 10:38 PM
This isn't open source, correct?
by freyr on 6/11/22, 6:42 AM
Uber has adopted Go (Golang for long)
by travisd on 6/10/22, 9:29 PM
(†famous last words, I know)
by jasonhansel on 6/11/22, 11:53 AM
It's not just data races--it's also logical races, which are near-impossible to detect or prevent without something like transactional memory.
by baalimago on 6/11/22, 6:16 AM
by sharno on 6/11/22, 8:49 PM
And if you feel that Erlang's lack of type safety is an issue, then Gleam has you covered.
by thallavajhula on 6/10/22, 9:17 PM
By system do you mean a process or a tool that detects these?
by ryanschneider on 6/11/22, 12:26 PM
Edit: oh I see it highlights red and underlines every keyword. I find that incredibly distracting, so much so I assumed their highlighter was broken, but also just realized they are screenshots.
by fmakunbound on 6/11/22, 1:46 AM
How is that possible and what do they do???
by JamesSwift on 6/10/22, 9:49 PM
by rapiz on 6/11/22, 4:46 AM
by gigatexal on 6/11/22, 7:37 AM
by fellellor on 6/11/22, 2:23 AM
by oconnor663 on 6/11/22, 10:15 PM
by smw1218 on 6/14/22, 8:35 PM
https://medium.com/@scott_white/concurrency-in-go-is-not-mag...
tl;dr Go doesn't magically solve data races and blaming the language itself isn't well supported by the examples/data.
by stevefan1999 on 6/11/22, 6:59 PM
by metadat on 6/10/22, 9:55 PM
The "Slices" example is just nasty! Like, this is just damning for Go's promise of "_relatively_ easy and carefree concurrency".
Think about it for a second or two,
>> The reference to the slice was resized in the middle of an append operation from another async routine.
What exactly happens in these cases? How can I trust myself, as a fallible human being, to reason about such cases when I'm trying to efficiently roll up a list of results. :-/
Compared to every other remotely mainstream language, perhaps even C++, these are extremely subtle and sharp.. nigh, razor sharp edges. Yuck.
One big takeaway is this harsh realization: Golang guarantees are scant more than what is offered by pure, relatively naïve and unadulterated BASH shell programming. I still will use it, but with newfound fear.
As a multi-hundred-kloc-authoring-gopher: I love Go, and this article is killing me inside. Go appears extremely sloppy at the edges of the envelope and language boundaries, moreso than even I had ever realized prior to now.
Full-disclosure: I am disgusted by the company that is Uber, but I'm grateful to the talented folks who've cast a light on this cesspool region of Golang. Thank you!
p.s. inane aside: I never would've guessed that in 2022, Java would start looking more and more appealing in new ways. Until now I've been more or less "all-in" on Go for years.