Skip to content
Spark Acceleration Compare

Ledger entry, published 2026-11-03

Comet 1.1.0 and 1.2.0 against Spark 4.2.0, TPC-DS SF1000

All Ledger entriesApache DataFusion Comet profile

Option tested
Apache DataFusion Comet
Ran by
Apache DataFusion Comet project (the maker of the option tested)
Published
2026-11-03
Workload
TPC-DS derived queries, 103 queries, Parquet (zstd) in S3, each query run twice, mean reported
Data size
1 TB (scale factor 1000)
Hardware
EKS cluster on r6i instances with EBS; 32 executors x 16 cores, 64G executor memory (Comet adds 32G off-heap), driver 128G
Result as published
Spark 4.2.0 994 s; Comet 1.1.0 601 s; Comet 1.2.0 488 s (published as 2.0x faster than Spark and 1.2x faster than Comet 1.1.0)
Cost result as published
None published
Method published
Yes, scripts in the Comet repository and the apache/datafusion-benchmarks repository

What the test shows and does not show

The derived queries are not comparable to official TPC-DS results, and the node count of the EKS cluster is not stated.

The speedup depends on the Spark baseline, as the same page's Spark 4.1.3 run shows.

Other Ledger entries for Apache DataFusion Comet

Sources

  1. Comet TPC-DS benchmark results, read 10 Nov 2026

Last reviewed 10 Nov 2026.