Skip to content
Spark Acceleration Compare

Glossary, read 10 Nov 2026

DataFusion

All termsOpen the Ledger

DataFusion is a toolkit for building engines, not a Spark replacement. Comet takes DataFusion's native operators and plugs them into Spark's execution path through the JVM.

Flarion, a commercial plugin, also says it is built on Arrow and DataFusion. The two products differ in license and support terms, so read each project's own documents.

A sourced example

The Comet project's TPC-DS run at scale factor 1000 on an EKS cluster reported 994 s on Spark 4.2.0 and 488 s on Comet 1.2.0; the project ran it, so read it as a project number.

Sources

  1. Apache DataFusion user guide: Introduction, read 10 Nov 2026

Last reviewed 10 Nov 2026.