Options, read 10 Nov 2026
12 ways to speed up Spark, listed A to Z with the same fields
Open the LedgerCompare two options
Every option on this site
| Option | Kind | Ledger entries | Summary |
|---|---|---|---|
| Amazon EMR runtime for Apache Spark | AWS-tuned Spark runtime on EMR | 1 | Spark runtime from AWS: 100% API compatible per AWS, on EMR on EC2, EKS and Serverless, with a public TPC-DS harness. |
| Apache Auron | Open-source Rust engine plugin for Spark | 1 | Apache-2.0 Spark plugin with a Rust native engine, a TPC-DS 1 TB method and builds for Spark 3.0 to 4.2. |
| Apache DataFusion Comet | Open-source Spark plugin on DataFusion | 2 | Apache-2.0 Spark plugin on DataFusion with published TPC-DS scripts, settings and monthly releases since 1.0.0. |
| Apache Gluten with Velox | Open-source Spark plugin offloading to Velox | 2 | Apache-2.0 Spark plugin offloading to Velox, with a built-in benchmark tool and a self-reported adopter list. |
| Databricks Photon | Databricks's C++ vectorized engine | 2 | Databricks's own C++ engine, on by default in classic jobs compute and documented only inside Databricks. |
| DataPelago | Gluten-based CPU and GPU accelerator | 1 | Proprietary Gluten-based Spark plugin for EKS, Dataproc and GKE, billed through cloud marketplaces. |
| DualBird | FPGA Spark plugin on AWS F2 | 3 | FPGA-based Spark plugin for AWS F2 instances, with one published worked example, an Iceberg compaction test. |
| Flarion | Rust and DataFusion engine as one JAR | 2 | Proprietary native Spark engine as one JAR, with published per-core list prices and a six-engine TPC-DS page. |
| Google Cloud Lightning Engine | Premium Spark engine on Google Cloud | 3 | Google Cloud's Spark engine tier with optional native execution, priced per DCU-hour in us-central1. |
| Microsoft Fabric native execution engine | Velox-based engine inside Fabric Spark | 0 | Velox and Gluten engine inside Fabric Spark on Azure, switched on with one toggle or one setting. |
| NVIDIA cuDF plugin for Apache Spark | GPU plugin, formerly RAPIDS Accelerator | 2 | GPU plugin for Spark SQL and DataFrame jobs, with a cloud-provider benchmark that publishes runtime and cost per run. |
| Speedata | PCIe analytics chip plus Dash Spark plugin | 2 | Custom-silicon Spark offload on a PCIe card, sold as servers, OEM hardware or a hosted AWS service. |
Last reviewed 10 Nov 2026.