Guide, read 10 Nov 2026
Spark acceleration options on Amazon EMR and EKS
What counts as running on EMR or EKS
Amazon EMR runs Spark three ways: EMR on EC2, EMR on EKS and EMR Serverless. EMR on EKS runs Spark pods on your Kubernetes cluster, with EMR images. A team on open-source Spark on EKS without EMR is a fourth case, and some projects document only that. The list below separates what each project documents from what it does not, and says none found on 2026-11-10 where a page is silent.
The options, in alphabetical order
Amazon EMR Spark runtime. AWS says the performance-optimized runtime runs up to 4.5x faster than open-source equivalents and gives 2.7x faster Iceberg writes, with no TPC-DS scale or cluster stated on the page. It is described as 100% API compatible with open-source Spark and proprietary to AWS, a tuned build of Spark rather than a native replacement. It is the only option here that needs no plugin, but jobs must run on EMR. AWS publishes the emr-spark-benchmark repository (3 TB TPC-DS, 104 queries, three iterations) with commands for open-source Spark, EMR on EC2, EMR Serverless and a link to EMR on EKS, and a companion emr-on-eks-benchmark repository. It does not publish results there.
Apache Auron. No EMR, EKS or Databricks support is documented. Its benchmark ran on YARN. The jar is built from source with Rust nightly.
Apache DataFusion Comet. Its published TPC-DS-derived results at scale factor 1000 ran on an EKS cluster using r6i instances with EBS and S3 data. Comet supports Spark 3.5.9, 4.0.4, 4.1.3 and 4.2.0 with JDK 17 or later, and its docs warn it may not fully work with cloud-provider Spark forks, so EMR's runtime is not documented as supported.
Apache Gluten with Velox. Its documentation does not mention EMR, EKS or Databricks support. Official Spark versions are 3.4.4, 3.5.5, 4.0.2 and 4.1.1; docs have pages for S3 storage.
DataPelago. On AWS, DPA-S is available for open-source Spark on EKS as a Docker container in ECR (Spark 3.5.1, Java 11, Scala 2.12, Ubuntu 22). It needs a subscription ID and a license bucket tied to registered AWS account IDs, and is billed through the AWS marketplace pay-as-you-go. EMR is not mentioned on its AWS page.
DualBird. DualBird says its plugin is compatible with Apache Spark, Amazon EMR and Amazon EKS, and that onboarding is switching the EC2 instance type to F2 and adding its plugin. It lists no Spark version numbers.
Flarion. Lists Amazon EMR (installed with a bootstrap action) and Spark on Kubernetes (Helm chart or Spark operator) among its platforms. It runs on the CPU fleet already in use. A supported-version list is not published. Pricing is $0.009 per CPU hour for self-hosted Spark on AWS, Azure or GCP and $0.012 for fully managed Spark including EMR.
NVIDIA cuDF plugin for Apache Spark. Amazon EMR on EKS supports the plugin from release 6.9.0, and the "-spark-rapids" release variant ships it pre-installed. NVIDIA's EMR guide lists the plugin jar each EMR release ships: EMR 7.13 and 7.12 carry Spark 3.5.6 with rapids-4-spark 25.08.0, EMR 7.0 carries Spark 3.5.0 with 23.10.0, and EMR 6.10 carries Spark 3.3.1 with 22.12.0. Needs GPU instances, such as the g7.4xlarge used in AWS's test.
Speedata. Not an instance family: the APU is a PCIe card, and the AWS option is a Speedata-hosted environment where the customer keeps data in S3 and results are pushed back. Speedata does not run inside the customer's EC2 fleet, and EMR and EKS support are not stated on the pages read.
Not available on EMR or EKS
Databricks Photon is documented only as part of Databricks compute. Microsoft Fabric's native execution engine runs only inside Fabric on Azure. Google's Lightning Engine is a Google Cloud service. A team that owns an EMR or EKS bill cannot add them without moving platforms.
What each one needs, side by side
DualBird's limits in one line: AWS F2 instances only, so not Azure, Google Cloud or Databricks-managed clusters, and every benchmark it has published is its own.
| Option | Instance or hardware | Spark versions documented | Billing note |
|---|---|---|---|
| Amazon EMR Spark runtime | Standard EMR instances | Per EMR release; Spark 4.0 and Iceberg v3 on the features page | EMR price added to EC2 and EBS; EMR on EKS per vCPU-hour and GB-hour requested, one-minute minimum |
| Apache DataFusion Comet | Any x86 or arm64 Linux; r6i in its benchmark | 3.5.9, 4.0.4, 4.1.3, 4.2.0 | Open source |
| DataPelago | EKS pods; hardware requirement not confirmed | 3.5.1 on AWS | Marketplace pay-as-you-go; unit price not published |
| DualBird | AWS F2 (f2.6xlarge to f2.48xlarge) | None found on 2026-11-10 | No plugin price published |
| Flarion | Existing CPU fleet | 3.5.5 in its benchmark; list not published | Per accelerated CPU hour |
| NVIDIA cuDF plugin | NVIDIA GPU instances, for example g7.4xlarge | 3.3.x to 4.2.0 across releases | Software free; GPU instance and EMR on EKS charges apply |
Questions to settle before a trial
Which EMR release label and Spark version you run today decides which options are even installable. Whether your jobs run on EC2 or EKS decides how the plugin reaches the executors: a bootstrap action, a custom image in ECR, or a release label. Which region you run in decides whether F2 or a given GPU family is offered. And whether the claimed saving includes the instance and EMR surcharge is something the vendor must state. The Change Budget records these as the cloud and cluster rows, and the guide on trialing an accelerator turns them into a plan.
Sources
- Amazon EMR: Apache Spark features, read 10 Nov 2026
- Amazon EMR pricing, read 10 Nov 2026
- AWS samples: emr-spark-benchmark, read 10 Nov 2026
- NVIDIA cuDF plugin: getting started on AWS EMR, read 10 Nov 2026
- AWS Big Data Blog: GPU-accelerated Apache Spark on Amazon EMR, read 10 Nov 2026
- Apache DataFusion Comet: installation, read 10 Nov 2026
- DataPelago: deploy on AWS, read 10 Nov 2026
- DualBird home page, read 10 Nov 2026
- Flarion product page, read 10 Nov 2026
- Flarion pricing, read 10 Nov 2026
- Speedata home page, read 10 Nov 2026
Last reviewed 10 Nov 2026.