Ledger entry, published 2026-11-03
Comet 1.1.0 and 1.2.0 against Spark 4.2.0, TPC-DS SF1000
All Ledger entriesApache DataFusion Comet profile
- Option tested
- Apache DataFusion Comet
- Ran by
- Apache DataFusion Comet project (the maker of the option tested)
- Published
- 2026-11-03
- Workload
- TPC-DS derived queries, 103 queries, Parquet (zstd) in S3, each query run twice, mean reported
- Data size
- 1 TB (scale factor 1000)
- Hardware
- EKS cluster on r6i instances with EBS; 32 executors x 16 cores, 64G executor memory (Comet adds 32G off-heap), driver 128G
- Result as published
- Spark 4.2.0 994 s; Comet 1.1.0 601 s; Comet 1.2.0 488 s (published as 2.0x faster than Spark and 1.2x faster than Comet 1.1.0)
- Cost result as published
- None published
- Method published
- Yes, scripts in the Comet repository and the apache/datafusion-benchmarks repository
What the test shows and does not show
The derived queries are not comparable to official TPC-DS results, and the node count of the EKS cluster is not stated.
The speedup depends on the Spark baseline, as the same page's Spark 4.1.3 run shows.
Other Ledger entries for Apache DataFusion Comet
- Comet 1.1.0 and 1.2.0 against Spark 4.1.3, TPC-DS SF10002026-11-03, run by Apache DataFusion Comet project
Sources
- Comet TPC-DS benchmark results, read 10 Nov 2026
Last reviewed 10 Nov 2026.