Guides, read 10 Nov 2026
Guides to speeding up Spark without rewriting jobs
Open the LedgerCompare two options
All 10 guides, A to Z
- GPU vs FPGA for Spark: the published facts on eachGPU versus FPGA for Apache Spark: what DualBird and NVIDIA have each published about speed, cost, hardware, operators and availability, side by side with sources.
- GPU, FPGA and custom silicon for Spark in plain wordsWhat FPGA, GPU and custom-silicon accelerators do for Apache Spark, with DualBird on AWS F2, NVIDIA cuDF and Speedata C200 as examples, plus AWS facts.
- How to read a vendor Spark benchmark: seven checksSeven checks for a Spark acceleration benchmark: who ran it, workload, scale factor, hardware, baseline Spark version, cost basis and published method.
- How to run a fair two-week Spark accelerator trialA two-week plan to trial a Spark accelerator on your own jobs: baseline, same data, same cluster cost, cost per job, event logs and the Spark UI, with sources.
- Native Spark engines: Auron, DataFusion, Photon, VeloxWhat vectorized native Spark engines do (Auron, DataFusion, Photon, Velox): columnar execution, operator coverage, fallback, Spark versions and where each runs.
- Operator coverage and fallback: checking what an accelerator coversHow to check which Spark operators an accelerator covers and what happens on fallback, using each project's supported-operator lists, explain output and the Spark UI.
- Spark acceleration options on Amazon EMR and EKSWhich Spark accelerators run on Amazon EMR and EKS and what each needs: instance families, plugins, EMR release labels, Spark versions and billing.
- Spark performance tuning: what to do when it is not enoughSpark performance tuning checklist (AQE, shuffle partitions, broadcast joins, file sizes, skew) and how to tell when tuning is not enough.
- What "no code change" means for Spark accelerationWhat no code change means for Spark acceleration: SQL and DataFrame versus RDD and UDF code, fallback, config flags and instance changes, per option.
- Where a Spark job spends its time: scan to JVMWhere Spark time goes: scan, shuffle, sort, join, spill, serialization and the JVM, with figures from the Berkeley NSDI 2015 study and the Apache Spark documentation.
Last reviewed 10 Nov 2026.