from Hacker News

Leader Election with S3 Conditional Writes

by gunnarmorling on 8/26/24, 1:54 PM with 70 comments

  • by mprime1 on 8/30/24, 2:05 PM

    Leader election and distributed locking reduce to the same problem… which is proven to be impossible. It means in some edge case it will fail on you, is your system handling those cases?

    I didn’t read past this:

    > Systems like Apache ZooKeeper or Postgres (via Advisory Locks) provide the required building blocks for this

    Zookeeper is the original sin. Convincing a whole generation of programmers that distributed lock are a feasible solution.

    This is my biggest pet peeve in distributed systems.

    ——

    And if you don’t believe me, maybe you’ll trust Kyle K. of Jepsen fame:

    > However, perfect failure detectors are impossible in asynchronous networks.

    Links to: https://www.cs.utexas.edu/~lorenzo/corsi/cs380d/papers/p225-...

    https://jepsen.io/analyses/datomic-pro-1.0.7075

  • by Spivak on 8/30/24, 3:28 PM

    Why are people going for this guys throat— distributed locking might be impossible but what is being described, distributed leasing, is totally possible and useful. There's no sins being committed here.

    I might choose DynamoDB over S3 to implement this but both are fine.

  • by deepsun on 8/31/24, 12:05 AM

    I did the same thing 5+ years ago on GCS, using generation id.

    And I thought S3 also supported that. I was always under impression that S3 at some point reached feature parity with GCS.

    Are there any other fundamental features they are not equal on?

  • by dekhn on 8/30/24, 7:24 PM

    Is chubby a filesystem?

  • by shayonj on 8/30/24, 4:18 PM

    Very cool case of using s3 conditional writes for distributed leasing

  • by mgdev on 8/30/24, 4:22 PM

    This is a very expensive way to do leader election, at least from an infrastructure perspective.

    This is because you're essentially pushing the problem down to S3, which does its own leader election in a way that is waaaay overbuilt for what we're trying to accomplish here.

    But... that doesn't mean it isn't cool. :)