from Hacker News

Apache Arrow DataFusion Is Now Apache DataFusion

by pauldix on 5/7/24, 1:17 PM with 1 comments

• by pauldix on 5/7/24, 1:21 PM

  DataFusion, a SQL parser, planner, optimizer, and execution engine written in Rust, is now a top level Apache project! This shows the continued growth and momentum of the project and community. We've been building InfluxDB v3 on DataFusion since the summer of 2020 and have been really happy with the number of external contributors and projects that have adopted it in that time.

  We believe it's going to be a foundational component of analytic and data systems of all kinds for the next 20 years. Recent developments like a team of Apple engineers open sourcing Comet, their Spark execution engine built on DataFusion show the strength of the project and the modular approach. It's a building block for more complex analytic systems.