Skip to content
Spark Acceleration Compare

Options, read 10 Nov 2026

12 ways to speed up Spark, listed A to Z with the same fields

Open the LedgerCompare two options

Every option on this site

Alphabetical by name.
OptionKindLedger entriesSummary
Amazon EMR runtime for Apache SparkAWS-tuned Spark runtime on EMR1Spark runtime from AWS: 100% API compatible per AWS, on EMR on EC2, EKS and Serverless, with a public TPC-DS harness.
Apache AuronOpen-source Rust engine plugin for Spark1Apache-2.0 Spark plugin with a Rust native engine, a TPC-DS 1 TB method and builds for Spark 3.0 to 4.2.
Apache DataFusion CometOpen-source Spark plugin on DataFusion2Apache-2.0 Spark plugin on DataFusion with published TPC-DS scripts, settings and monthly releases since 1.0.0.
Apache Gluten with VeloxOpen-source Spark plugin offloading to Velox2Apache-2.0 Spark plugin offloading to Velox, with a built-in benchmark tool and a self-reported adopter list.
Databricks PhotonDatabricks's C++ vectorized engine2Databricks's own C++ engine, on by default in classic jobs compute and documented only inside Databricks.
DataPelagoGluten-based CPU and GPU accelerator1Proprietary Gluten-based Spark plugin for EKS, Dataproc and GKE, billed through cloud marketplaces.
DualBirdFPGA Spark plugin on AWS F23FPGA-based Spark plugin for AWS F2 instances, with one published worked example, an Iceberg compaction test.
FlarionRust and DataFusion engine as one JAR2Proprietary native Spark engine as one JAR, with published per-core list prices and a six-engine TPC-DS page.
Google Cloud Lightning EnginePremium Spark engine on Google Cloud3Google Cloud's Spark engine tier with optional native execution, priced per DCU-hour in us-central1.
Microsoft Fabric native execution engineVelox-based engine inside Fabric Spark0Velox and Gluten engine inside Fabric Spark on Azure, switched on with one toggle or one setting.
NVIDIA cuDF plugin for Apache SparkGPU plugin, formerly RAPIDS Accelerator2GPU plugin for Spark SQL and DataFrame jobs, with a cloud-provider benchmark that publishes runtime and cost per run.
SpeedataPCIe analytics chip plus Dash Spark plugin2Custom-silicon Spark offload on a PCIe card, sold as servers, OEM hardware or a hosted AWS service.

Last reviewed 10 Nov 2026.