# Comet 1.1.0 and 1.2.0 against Spark 4.2.0, TPC-DS SF1000

Ledger entry, published 2026-11-03

**The short answer.** The Apache DataFusion Comet project ran 103 TPC-DS derived queries at scale factor 1000 on an EKS cluster of r6i instances. Spark 4.2.0 took 994 seconds, Comet 1.1.0 took 601 and Comet 1.2.0 took 488. The project publishes scripts and its non-default Spark settings.

- **Option tested:** [Apache DataFusion Comet](/options/datafusion-comet/)
- **Ran by:** Apache DataFusion Comet project (the maker of the option tested)
- **Published:** 2026-11-03
- **Workload:** TPC-DS derived queries, 103 queries, Parquet (zstd) in S3, each query run twice, mean reported
- **Data size:** 1 TB (scale factor 1000)
- **Hardware:** EKS cluster on r6i instances with EBS; 32 executors x 16 cores, 64G executor memory (Comet adds 32G off-heap), driver 128G
- **Result as published:** Spark 4.2.0 994 s; Comet 1.1.0 601 s; Comet 1.2.0 488 s (published as 2.0x faster than Spark and 1.2x faster than Comet 1.1.0)
- **Cost result as published:** None published
- **Method published:** Yes, scripts in the Comet repository and the apache/datafusion-benchmarks repository

## What the test shows and does not show

The derived queries are not comparable to official TPC-DS results, and the node count of the EKS cluster is not stated.

The speedup depends on the Spark baseline, as the same page's Spark 4.1.3 run shows.

## Other Ledger entries for Apache DataFusion Comet

- [Comet 1.1.0 and 1.2.0 against Spark 4.1.3, TPC-DS SF1000](/ledger/datafusion-comet-tpcds-sf1000-vs-spark-4-1-3/): 2026-11-03, run by Apache DataFusion Comet project

## Sources

- Comet TPC-DS benchmark results: https://raw.githubusercontent.com/apache/datafusion-comet/main/docs/source/contributor-guide/benchmark-results/tpc-ds.md (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://sparkaccelerationcompare.com/ledger/datafusion-comet-tpcds-sf1000-vs-spark-4-2-0/
