Skip to content
Spark Acceleration Compare

Side by side, read 10 Nov 2026

Apache Gluten with Velox and NVIDIA cuDF plugin for Apache Spark: the published facts in the same rows

Apache Gluten with Velox profileNVIDIA cuDF plugin for Apache Spark profile

The same rows for both

Options in alphabetical order. Vendor statements are the vendor's own. Read 10 Nov 2026.
RowApache Gluten with VeloxNVIDIA cuDF plugin for Apache Spark
KindOpen-source Spark plugin offloading to VeloxGPU plugin, formerly RAPIDS Accelerator
Does wellGluten needs no DataFrame or SQL changes, only configuration: the GlutenPlugin class, off-heap memory and the columnar shuffle manager. The repository includes the gluten-it tool for data generation, query runs and result comparison on Spark 3.4, 3.5, 4.0 and 4.1 profiles.AWS's August 2026 post publishes runtime and cost per run: 281 seconds and $2.06 on GPU versus 1,010 to 1,043 seconds and $2.93 to $3.18 on CPU clusters. Amazon EMR on EKS supports the plugin natively from release 6.9.0, and NVIDIA lists Databricks 14.3 ML LTS and 17.3 ML LTS and Dataproc 2.1 to 2.3 as supported.
LimitedThe docs name Spark 3.4.4, 3.5.5, 4.0.2 and 4.1.1 as officially supported and warn that an unofficial Spark version can throw NoSuchMethodError at runtime. Per the Velox backend docs, Iceberg support is read-only, Hudi support covers copy-on-write tables only, and ANSI mode always falls back to vanilla Spark.The plugin needs NVIDIA GPUs of the Volta generation or newer, so the cluster moves to GPU instances such as g7.4xlarge. Only the DataFrame API and a limited set of operations are accelerated; RDDs, GraphX and APIs that take Scala or Java functions are not.
Published speed resultsThe project homepage reports TPC-H and TPC-DS at 3 TB on Intel Xeon Platinum 8592+ with Spark 3.3.1: overall 3.34x on TPC-H (up to 23.45x on single queries) and 3.02x on TPC-DS (up to 13.75x on single queries). The page does not link a method or scripts for these numbers; the hardware is from Intel, a contributor.AWS and NVIDIA report that TPC-DS at 3 TB (103 queries) on 8-node Amazon EMR on EKS clusters ran in 281 seconds on g7.4xlarge GPU instances (NVIDIA RTX PRO 4500 Blackwell) versus 1,010 to 1,043 seconds on comparable CPU instances, which AWS labels a 3.7x speedup. Median of 5 iterations, Spark 3.5.6, cuDF plugin 26.04.2, us-east-1. NVIDIA's own FAQ gives a general expectation and says there is no formula for a specific query. This is a vendor claim with no named benchmark behind it. NVIDIA's overview page compares the published AWS GPU result with separate NDS (TPC-DS derived) 3 TB runs of open-source Spark 3.5.6 on EC2. It says the GPU run is 8.4x faster than m7gd.8xlarge and 11.0x faster than m9gd.4xlarge, with 77% and 72% lower reported cost per run. NVIDIA itself notes the runs use different hardware and software stacks.
Published cost results and pricesNone found on 10 Nov 2026.AWS reports cost per benchmark run of $2.06 on the GPU cluster versus $2.93 to $3.18 on the CPU clusters, using On-Demand us-east-1 prices as of May 2026. The g7.4xlarge cluster costs $26.35 per hour (8 x $3.042 EC2 plus $2.01 for EMR on EKS) against $10.43 to $10.99 per hour for the CPU clusters. Cost excludes the EKS control-plane fee, EBS, S3 and the driver pod. The software itself is free; you pay for GPU instances.
Code changesGluten is a Spark plugin that needs no DataFrame or SQL changes, only configuration: spark.plugins=org.apache.gluten.GlutenPlugin, off-heap memory enabled with a size, ColumnarShuffleManager and the bundle jar. Unsupported operators fall back to vanilla Spark through columnar-to-row and row-to-columnar conversions. The Velox backend limitations page lists known fallbacks and differences: ANSI mode always falls back, case-sensitive mode may give wrong results, regexp functions use RE2 (no lookaround), file scans fully support only Parquet and partly ORC, partitioned scans need partition info in the path, and JSON functions differ for single-quoted strings.NVIDIA says existing Spark applications run with no code change once the plugin is launched and enabled by a configuration setting. Unsupported operators and UDFs fall back to the CPU, so the job completes but those stages are not accelerated. Only the DataFrame API (Dataset[Row]) and a limited set of SQL and DataFrame operations are accelerated. Custom classes or types with Dataset, APIs that take Row or Scala/Java functions (flatMap, foreach, foreachPartition), GraphX and RDDs are not accelerated. Structured streaming is not a focus.
TuningOff-heap memory must be enabled and sized (the quick start uses spark.memory.offHeap.size=20g and says to adjust it for the environment), and the columnar shuffle manager is needed for native shuffle. Docs also describe a troubleshooting page, Celeborn and Uniffle remote shuffle settings, and a setting (spark.gluten.sql.fallbackRegexpExpressions) for users who cannot tolerate regexp differences. Binary jars must load ahead of Spark's jars because Gluten overrides some Spark classes.The user guide has a tuning guide (executors, tasks per executor, pooled and pinned memory, concurrent tasks per GPU, shuffle partitions, input partition size, columnar batch size), an AutoTuner, a Profiling tool and a Qualification tool, plus a recommended-practices section on handling GPU out-of-memory errors. In the AWS test the GPU config set executor memoryOverhead 30G, pinnedPool 8G, concurrentGpuTasks 3 and RapidsShuffleManager, and AWS advises using the Spark History Server to tune concurrentGpuTasks and pinnedPool.size. AWS recommends a pre-migration assessment with the NVIDIA qualification tool to estimate GPU acceleration potential, and setting spark.rapids.sql.explain=NOT_ON_GPU to see which operations run on CPU.
Spark versions, platforms and table formatsThe docs state officially supported Spark versions are 3.4.4, 3.5.5, 4.0.2 and 4.1.1 (Spark 4.x needs JDK 17+ and Scala 2.13). The downloads page still lists Spark 3.3 binaries for 1.6.0 and 1.7.0, and the homepage benchmark used Spark 3.3.1. Using an unofficial Spark version can throw NoSuchMethodError at runtime. Docs do not mention EMR, EKS or Databricks support. Velox backend docs: Iceberg reads of copy-on-write and merge-on-read tables are supported but the Iceberg page says only reading is supported and only Spark 3.4 is well tested; Hudi supports reading COW tables only; Paimon supports non-primary-key tables only; Delta has a build profile and a doc page. Celeborn 0.3 to 0.5 and Uniffle 0.9.2 remote shuffle are supported.Release v26.10.0 supports Apache Spark 3.3.x, 3.4.x, 3.5.0 to 3.5.9, 4.0.0 to 4.0.4, 4.1.1 to 4.1.3 and 4.2.0 (Scala 2.12 and 2.13 depending on version). Supported Databricks runtimes on AWS and Azure are 14.3 ML LTS (Spark 3.5.0) and 17.3 ML LTS (Spark 4.0.0). Dataproc 2.1 to 2.3 and Dataproc Serverless runtimes 1.2, 2.2 and 3.0 are supported. Amazon EMR on EKS natively supports the plugin (release 6.9.0 or later; the '-spark-rapids' release variant ships it pre-installed). The user guide also has a getting-started page for AWS EMR (EC2) and covers Apache Iceberg and Delta Lake support pages, including reading and writing tables.
Where it runsOfficial build and test platform is Ubuntu 20.04/22.04 or CentOS 8; the downloads page says binaries were built on CentOS 7 with static linking on Intel Xeon Platinum 8272CL class CPUs, and that performance and compatibility are not guaranteed on other OSes or hardware, so building from source is recommended. Docs have pages for S3, GCS and ABFS storage.Needs NVIDIA GPUs: Volta, Turing, Ampere, Ada Lovelace, Hopper and Blackwell generations; tested on V100, T4, A10, A100, L4, H100, B100, RTX PRO 4500 and RTX PRO 6000. Linux with glibc 2.28 or newer; NVIDIA driver R525+ (CUDA 12) or R580+ (CUDA 13). On AWS the benchmark used g7.4xlarge (1 GPU, 16 vCPU, 64 GB RAM); NVIDIA's EMR guide maps m5d.8xlarge to g4dn.8xlarge for T4 GPUs.
How open its benchmarks areThe repo includes the gluten-it integration test and benchmark tool (data generation, query runs, result comparison against a baseline) for building and running TPC-style queries locally, supported on Spark 3.4, 3.5, 4.0 and 4.1 profiles. The homepage TPC-H and TPC-DS numbers do not themselves link to a run configuration.NVIDIA publishes the NDS benchmark automation (data generation, power and throughput runs) in a public Apache-2.0 repo, with a disclaimer that NDS is derived from TPC-DS and is not comparable to published TPC-DS results. The AWS post publishes the cluster, Spark config, EMR release and pricing method; it does not link scripts.
Change Budget: codeNo DataFrame or SQL changes. Gluten needs no DataFrame or SQL changes, only configuration.No code change stated. NVIDIA says existing Spark applications run with no code change once the plugin is launched and enabled by a configuration setting. Unsupported operators and UDFs fall back to the CPU.
Change Budget: clusterPlugin, off-heap memory, shuffle manager. The settings are spark.plugins=org.apache.gluten.GlutenPlugin, off-heap memory enabled with a size, ColumnarShuffleManager and the bundle jar loaded ahead of Spark's jars.GPU instances and plugin settings. The cluster needs NVIDIA GPUs (Volta to Blackwell generations) and a current driver. AWS's test used g7.4xlarge nodes with executor memoryOverhead 30G, pinnedPool 8G, concurrentGpuTasks 3 and RapidsShuffleManager.
Change Budget: cloudNone found on 2026-11-10. No cloud requirement is stated. Official build and test platforms are Ubuntu 20.04/22.04 or CentOS 8, and the docs have pages for S3, GCS and ABFS storage.AWS, Azure and Google Cloud documented. NVIDIA lists Databricks runtimes on AWS and Azure and Dataproc on Google Cloud; Amazon EMR on EKS supports the plugin natively from release 6.9.0.
Change Budget: vendorApache-2.0, no contract stated. Gluten's project license is Apache-2.0; no vendor or platform contract is described.Free software; GPU instances paid. The software is free (source Apache-2.0); you pay for GPU instances, and AWS's cost per run includes EC2 and EMR on EKS charges.
Ledger entriesGluten with Velox, TPC-DS 3 TB (project homepage) (run by its maker); Gluten with Velox, TPC-H 3 TB (project homepage) (run by its maker)GPU Spark on Amazon EMR on EKS with G7 instances, TPC-DS 3 TB (run by its maker); NVIDIA comparison with open-source Spark on EC2, NDS 3 TB (run by its maker)

How to read this page

The rows hold what each vendor or a named third party published, with no conclusion drawn. Benchmarks in the two columns were run on different hardware, data and baselines unless the Ledger entry says otherwise, so the numbers are not comparable across columns. The Ledger method explains how entries are recorded.

Sources

  1. Apache Gluten with the Velox backend: gluten.apache.org, read 10 Nov 2026
  2. Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-st, read 10 Nov 2026
  3. Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/velox-b, read 10 Nov 2026
  4. Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta, read 10 Nov 2026
  5. Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta, read 10 Nov 2026
  6. Apache Gluten with the Velox backend: gluten.apache.org/downloads, read 10 Nov 2026
  7. Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/tools/gluten-it/README.md, read 10 Nov 2026
  8. NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): aws.amazon.com/blogs/big-data/gpu-accelerated-apache-spark-with-amazon-e, read 10 Nov 2026
  9. NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): docs.nvidia.com/spark-rapids/user-guide/latest/faq.html, read 10 Nov 2026
  10. NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): docs.nvidia.com/spark-rapids/user-guide/latest/overview.html, read 10 Nov 2026
  11. NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): docs.nvidia.com/spark-rapids/user-guide/latest/index.html, read 10 Nov 2026
  12. NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): nvidia.github.io/cudf-spark/docs/download.html, read 10 Nov 2026
  13. NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): raw.githubusercontent.com/NVIDIA/cudf-spark-benchmarks/dev/nds/README.md, read 10 Nov 2026
  14. Apache Gluten Powered By page, read 10 Nov 2026

Last reviewed 10 Nov 2026.