Skip to content
Spark Acceleration Compare

Guide, read 10 Nov 2026

How to run a fair two-week trial on your own jobs

All guidesOpen the Ledger

What the trial has to answer

Published benchmarks answer a question about someone else's workload. A trial answers one question about yours: for the same jobs on the same data, what does a finished job cost with the accelerator, and what does it cost without? Everything below is there to keep the two sides comparable. The plan covers 14 days: week one for the baseline and setup, week two for the accelerated runs and the comparison.

The trial does not decide anything for you. It produces a table of your own numbers that you can set beside the published ones in the Spark Accelerator Ledger.

Week one: baseline, data and logs

Choose the jobs. Pick five to ten jobs that between them account for most of the bill, including at least one that runs late. Use the stage split from the guide on where Spark time goes to see which ones are scan, shuffle or compute heavy. Check each against the options' definitions of no code change and the operator lists.

Tune the baseline first. The baseline should be the configuration you would run in production today, not the one from last year. Check AQE, shuffle partitions, broadcast thresholds and small files, as listed in the tuning guide. The Comet project's own page shows why: the same engine looked 2.0x or 2.51x faster depending on which Spark version it was compared with.

Freeze the data. Run on a fixed snapshot of input tables, or on one day's data replayed, so both sides read the same bytes. A moving table breaks the comparison.

Turn on event logs. spark.eventLog.enabled defaults to false, so set it to true with a log directory before the first run, and run the Spark History Server so the UI can be rebuilt after the application ends. For long applications the documentation points to rolling event logs (spark.eventLog.rolling.enabled and maxFileSize) to avoid one huge file. Keep the logs for every run, baseline and trial.

Repeat the baseline. Run each baseline job several times across different days. AWS and NVIDIA report a median of five iterations; the Comet project runs each query twice and reports the mean; Flarion uses two runs at 100 GB and one at 1 TB. Your own variation from noisy neighbors, S3 latency and spot interruptions is the floor under any claimed gain.

Week two: same cluster cost, then the accelerated runs

Decide the unit before you start. The fair comparison is cost per finished job, not seconds. Fix one of two things: the same cluster cost per hour (so a faster job is cheaper) or the same job output at whatever cluster the accelerator needs. Do not run an accelerated job on a bigger cluster and compare runtimes.

Record per run: wall-clock time, instance types and counts, hours billed, and any license or service fee. Billing details differ. EMR on EKS charges per vCPU-hour and GB-hour requested, from image download until pod end, rounded to the second, with a one-minute minimum, plus EKS and EC2. On Databricks, Photon instance types carry their own DBU multiplier, so log it per run. DualBird publishes no plugin price, so ask for one in writing before the trial starts. Flarion bills per accelerated CPU hour. Google bills Lightning Engine at a premium rate ($0.089 per DCU-hour against $0.06 standard in the pricing page's us-central1 example). Speedata publishes no price list, so ask it too. For NVIDIA, the AWS post states its method: cost per run is cluster hourly cost times run time, with the excluded items listed.

Cost per job = (sum of instance hours times hourly price, plus platform and license fees) for the job's runs. Write the formula in your results sheet and fill it from billing records, not from estimates.

What to read in the Spark UI and event logs

For each run, compare the stage durations, shuffle read and write sizes, spill and GC time. On the SQL tab, count how many operators ran natively and how many fell back. Native engines show the evidence in different places, listed in the guide on operator coverage. A job that shows a large speedup on three stages and none on the fourth is telling you what the fourth contains.

Run the vendor's estimator on the baseline logs before the trial if one exists. NVIDIA's Qualification tool analyzes CPU event logs and predicts GPU duration and speedup with an XGBoost model trained on paired CPU and GPU runs. Speedata's Workload Analyzer works from uploaded Spark logs, and Flarion offers a free assessment that simulates each job from logs. Compare their predictions with what you measure. The gap is itself a finding.

The results sheet and the questions it must answer

Add two checks the published tests rarely cover: correctness (compare output row counts and checksums with the baseline; projects list differences, such as regexp semantics or ANSI mode) and operations (what it takes to upgrade Spark, restart executors and debug a failure with the plugin on).

Then stop at what you measured. A two-week result on ten jobs is evidence about those ten jobs. Whether it applies to the rest depends on how similar they are, which the stage splits and fallback counts let you judge.

One row per job.
ColumnWhy it is there
Job, input snapshot, Spark versionProves both sides ran the same thing
Baseline cost per job (median of N runs)The number to beat, after tuning
Trial cost per job (median of N runs)Same formula, same prices
Stages native versus fallbackExplains the result and predicts the next job
Failures, wrong results, config changes neededThe cost of running it in production
Fees and prices not yet quotedItems still open, such as a plugin price

Sources

  1. Apache Spark documentation: Monitoring and instrumentation, read 10 Nov 2026
  2. Apache Spark documentation: Configuration, read 10 Nov 2026
  3. Apache Spark documentation: Web UI, read 10 Nov 2026
  4. Amazon EMR pricing, read 10 Nov 2026
  5. AWS Big Data Blog: GPU-accelerated Apache Spark on Amazon EMR, read 10 Nov 2026
  6. NVIDIA cuDF plugin: Qualification tool, read 10 Nov 2026
  7. Apache DataFusion Comet: TPC-DS benchmark results, read 10 Nov 2026
  8. Google Cloud: Managed Service for Apache Spark pricing, read 10 Nov 2026
  9. Flarion pricing, read 10 Nov 2026
  10. Speedata Workload Analyzer, read 10 Nov 2026

Last reviewed 10 Nov 2026.