from Hacker News

Onyx: fault tolerant data processing for Clojure

by coding4all on 3/6/15, 9:12 AM with 21 comments

• by afandian on 3/6/15, 11:30 AM

  This looks very interesting. I'm doing some log file processing in Apache Spark in Clojure. Spark is written in Scala, but has a Java API, which is wrapped by Flambo. It looks and feels entirely Clojure.

  The semantics look very similar indeed. Does anyone have a comparison between Onyx and Spark?

• by XPherior on 3/6/15, 3:16 PM

  Hi folks! I'm Michael Drogalis - the primary author. I'm happy to answer any questions.

• by johnmurray_io on 3/6/15, 1:28 PM

  Checkout the original video introducing Onyx: http://youtu.be/vG47Gui3hYE

• by lkrubner on 3/6/15, 4:03 PM

  If this interests you, then you should also check out the post where Michael Drogalis first introduced this:

  http://michaeldrogalis.tumblr.com/post/98143185776/onyx-dist...

• by dj-wonk on 3/6/15, 5:53 PM

  Re: Onyx's architecture. I would wonder about performance when keeping a shared log in ZooKeeper. Why not use something like Kafka -- it is designed for high-volume, immutable logging. ZK works best for less-frequently changing configuration, such as node connection information or snapshotting. I could be wrong. I'd like to hear your thoughts and experience.

• by boothead on 3/6/15, 2:23 PM

  Looks superficially simmilar to https://github.com/aphyr/tesser anyone know both and can give a comparison?

  From a brief examination tesser looks a lot simpler (probably because of encoding most of the folding using various monoids). Does onyx have a similar abstraction model that I missed?