# Comet 1.1.0 and 1.2.0 against Spark 4.1.3, TPC-DS SF1000

Ledger entry, published 2026-11-03

**The short answer.** The Comet project reran its TPC-DS scale factor 1000 test against Spark 4.1.3 on the same EKS cluster. Spark took 1,336 seconds, Comet 1.1.0 took 613 (2.18x) and Comet 1.2.0 took 532 (2.51x). The project says Spark 4.1.3 is slower mainly because of the default S3A input stream.

- **Option tested:** [Apache DataFusion Comet](/options/datafusion-comet/)
- **Ran by:** Apache DataFusion Comet project (the maker of the option tested)
- **Published:** 2026-11-03
- **Workload:** TPC-DS derived queries, 103 queries, Parquet (zstd) in S3
- **Data size:** 1 TB (scale factor 1000)
- **Hardware:** Same EKS cluster on r6i instances as the Spark 4.2.0 run
- **Result as published:** Spark 4.1.3 1,336 s; Comet 1.1.0 613 s (2.18x); Comet 1.2.0 532 s (2.51x)
- **Cost result as published:** None published
- **Method published:** Yes, scripts in the Comet repository and the apache/datafusion-benchmarks repository

## What the test shows and does not show

Spark 4.2.0 with the classic S3A stream took 1,294 s on the same cluster, according to the project.

The two baseline rows together show how far a speedup ratio moves with the Spark version.

## Other Ledger entries for Apache DataFusion Comet

- [Comet 1.1.0 and 1.2.0 against Spark 4.2.0, TPC-DS SF1000](/ledger/datafusion-comet-tpcds-sf1000-vs-spark-4-2-0/): 2026-11-03, run by Apache DataFusion Comet project

## Sources

- Comet TPC-DS benchmark results: https://raw.githubusercontent.com/apache/datafusion-comet/main/docs/source/contributor-guide/benchmark-results/tpc-ds.md (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://sparkaccelerationcompare.com/ledger/datafusion-comet-tpcds-sf1000-vs-spark-4-1-3/
