# Apache Gluten with Velox and DataPelago: the published facts in the same rows

Side by side, read 10 Nov 2026

**The short answer.** Apache Gluten with Velox and DataPelago are set out here in the same rows: published speed and cost results, code and cluster changes, tuning, stack fit, where each runs and how open the tests are. The Ledger holds 2 published entries for Apache Gluten with Velox and 1 entry for DataPelago, each with who ran it.

## The same rows for both

| Row | Apache Gluten with Velox | DataPelago |
| --- | --- | --- |
| Kind | Open-source Spark plugin offloading to Velox | Gluten-based CPU and GPU accelerator |
| Does well | Gluten needs no DataFrame or SQL changes, only configuration: the GlutenPlugin class, off-heap memory and the columnar shuffle manager. The repository includes the gluten-it tool for data generation, query runs and result comparison on Spark 3.4, 3.5, 4.0 and 4.1 profiles. | Per DataPelago's documentation, DPA-S runs in the customer's own cloud account on EKS, Dataproc 2.2.49 with Spark 3.5.1, Dataproc 2.1.75 with Spark 3.3.2, and GKE. DataPelago's blog publishes operator microbenchmarks of Nucleus v3.0 against NVIDIA cuDF 25.04.00 on one H100 GPU of an AWS p5.48xlarge. |
| Limited | The docs name Spark 3.4.4, 3.5.5, 4.0.2 and 4.1.1 as officially supported and warn that an unofficial Spark version can throw NoSuchMethodError at runtime. Per the Velox backend docs, Iceberg support is read-only, Hudi support covers copy-on-write tables only, and ANSI mode always falls back to vanilla Spark. | DataPelago's 10X and 80% claims, and its customer quotes of 50% lower cost, carry no baseline, workload or method. Pricing is pay-as-you-go through the AWS and GCP marketplaces, and no unit price is published on the pages read. The AWS docs describe EKS with Spark 3.5.1 only, and EMR and Databricks are not mentioned. |
| Published speed results | The project homepage reports TPC-H and TPC-DS at 3 TB on Intel Xeon Platinum 8592+ with Spark 3.3.1: overall 3.34x on TPC-H (up to 23.45x on single queries) and 3.02x on TPC-DS (up to 13.75x on single queries). The page does not link a method or scripts for these numbers; the hardware is from Intel, a contributor. | The home page headline claims 10X faster at 80% cost reduction for GenAI and analytics data processing. The DPA-S page says it can reduce costs by up to 80% on qualifying workloads without rewriting applications or migrating data. No workload, baseline or method is attached to the 10X/80% figures. DataPelago's blog (Aug 21, 2025) compares Nucleus v3.0 with NVIDIA cuDF 25.04.00 on one H100 of an AWS p5.48xlarge, for single operators. Nucleus is up to 10.5x faster on project, up to 10.1x on filter, up to 4.3x on aggregate, up to 4.5x on hash aggregate, and up to 8.2x on Top-K. These are operator microbenchmarks against a GPU library, not end-to-end Spark jobs. |
| Published cost results and prices | None found on 10 Nov 2026. | Named customer quotes on the vendor site: Akad Seguros (CTO) says costs fell by more than 50%; ShareChat (VP of Data) says the heavy OLAP cube jobs now complete on OSS Spark and costs fell by 50%; RevSure.ai (CEO) reports gains in days with zero application changes; McAfee and Samsung SDS America describe 'promising results' only. These are testimonials with no baseline, workload or method. The DPA-S page describes pay-as-you-go pricing billed through the AWS and GCP marketplaces ('Use more and Save more'). No price per unit is published on the pages read. The docs require a Subscription ID and license bucket path tied to registered AWS account IDs. |
| Code changes | Gluten is a Spark plugin that needs no DataFrame or SQL changes, only configuration: spark.plugins=org.apache.gluten.GlutenPlugin, off-heap memory enabled with a size, ColumnarShuffleManager and the bundle jar. Unsupported operators fall back to vanilla Spark through columnar-to-row and row-to-columnar conversions. The Velox backend limitations page lists known fallbacks and differences: ANSI mode always falls back, case-sensitive mode may give wrong results, regexp functions use RE2 (no lookaround), file scans fully support only Parquet and partly ORC, partitioned scans need partition info in the path, and JSON functions differ for single-quoted strings. | DPA-S is presented as working with existing Spark clusters and jobs with no application rewrites or data migration; languages are PySpark, Scala and Spark SQL. Workloads to select: compute-intensive Parquet/ORC SQL and DataFrame jobs. Workloads to avoid: heavy use of RDDs and UDFs. The docs list limitations: native CPU acceleration is not used when ANSI mode is on (queries run on the JVM), case-sensitive mode is not supported and may return incorrect results, plus listed caveats for regexp functions, file source formats, JSON functions and NaN. |
| Tuning | Off-heap memory must be enabled and sized (the quick start uses spark.memory.offHeap.size=20g and says to adjust it for the environment), and the columnar shuffle manager is needed for native shuffle. Docs also describe a troubleshooting page, Celeborn and Uniffle remote shuffle settings, and a setting (spark.gluten.sql.fallbackRegexpExpressions) for users who cannot tolerate regexp differences. Binary jars must load ahead of Spark's jars because Gluten overrides some Spark classes. | Mandatory Spark settings in the EKS docs include spark.plugins=org.apache.gluten.GlutenPlugin, the subscription ID and license location env vars, spark.memory.offHeap.enabled=true with a positive spark.memory.offHeap.size, spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager and netty Java options. CBO settings are recommended. The DPA-S page tells users to adjust Spark configuration (shuffle manager, memory tuning). So some per-deployment configuration is required, in contrast to 'no tuning' claims by other vendors. |
| Spark versions, platforms and table formats | The docs state officially supported Spark versions are 3.4.4, 3.5.5, 4.0.2 and 4.1.1 (Spark 4.x needs JDK 17+ and Scala 2.13). The downloads page still lists Spark 3.3 binaries for 1.6.0 and 1.7.0, and the homepage benchmark used Spark 3.3.1. Using an unofficial Spark version can throw NoSuchMethodError at runtime. Docs do not mention EMR, EKS or Databricks support. Velox backend docs: Iceberg reads of copy-on-write and merge-on-read tables are supported but the Iceberg page says only reading is supported and only Spark 3.4 is well tested; Hudi supports reading COW tables only; Paimon supports non-primary-key tables only; Delta has a build profile and a doc page. Celeborn 0.3 to 0.5 and Uniffle 0.9.2 remote shuffle are supported. | On AWS, DPA-S is available for OSS Spark on EKS as a Docker container in ECR: Spark 3.5.1, Java 11, Scala 2.12, Ubuntu 22. EMR and Databricks are not mentioned on the AWS docs page. The vendor page says 'multiple Spark versions on Google Cloud and AWS'. Lakehouse formats Iceberg, Hudi and Delta are named only in a reseller testimonial (Twingo) on the vendor site. GCP docs list supported combinations: Dataproc 2.2.49 with Spark 3.5.1, Dataproc 2.1.75 with Spark 3.3.2, OSS Spark 3.5.1 and OSS Spark 3.3.4 (all Java 11, Scala 2.12) and a GKE deployment. Docs version header is 3.1.7. The technology page says the engine builds on Substrait-based open-source frameworks such as Apache Gluten and Velox; the docs configure the Gluten plugin class. The Gluten initiator is quoted on the home page in support. Nucleus and DPA-S themselves are proprietary and licensed. |
| Where it runs | Official build and test platform is Ubuntu 20.04/22.04 or CentOS 8; the downloads page says binaries were built on CentOS 7 with static linking on Intel Xeon Platinum 8272CL class CPUs, and that performance and compatibility are not guaranteed on other OSes or hardware, so building from source is recommended. Docs have pages for S3, GCS and ABFS storage. | Runs in the customer's own cloud account on AWS (EKS) and Google Cloud (Dataproc, GKE). The Nucleus engine targets CPU, GPU and FPGA. The Nucleus-versus-cuDF test used an AWS p5.48xlarge with H100 GPUs; the DPA-S docs pages read do not state a GPU requirement, so the hardware needed for DPA-S itself was not confirmed. |
| How open its benchmarks are | The repo includes the gluten-it integration test and benchmark tool (data generation, query runs, result comparison against a baseline) for building and running TPC-style queries locally, supported on Spark 3.4, 3.5, 4.0 and 4.1 profiles. The homepage TPC-H and TPC-DS numbers do not themselves link to a run configuration. | None found on 10 Nov 2026. |
| Change Budget: code | No DataFrame or SQL changes. Gluten needs no DataFrame or SQL changes, only configuration. | No application rewrite stated. DataPelago presents DPA-S as working with existing Spark clusters and jobs with no application rewrites or data migration. It tells users to avoid heavy use of RDDs and UDFs. |
| Change Budget: cluster | Plugin, off-heap memory, shuffle manager. The settings are spark.plugins=org.apache.gluten.GlutenPlugin, off-heap memory enabled with a size, ColumnarShuffleManager and the bundle jar loaded ahead of Spark's jars. | Plugin, license and memory settings. Mandatory settings include the Gluten plugin class, the subscription ID and license location, off-heap memory, the columnar shuffle manager and Netty Java options. On AWS it ships as a Docker container in ECR for Spark 3.5.1 on EKS. |
| Change Budget: cloud | None found on 2026-11-10. No cloud requirement is stated. Official build and test platforms are Ubuntu 20.04/22.04 or CentOS 8, and the docs have pages for S3, GCS and ABFS storage. | AWS EKS or Google Cloud. DataPelago documents EKS on AWS and Dataproc and GKE on Google Cloud, in the customer's own account. |
| Change Budget: vendor | Apache-2.0, no contract stated. Gluten's project license is Apache-2.0; no vendor or platform contract is described. | Marketplace subscription required. DPA-S is billed pay-as-you-go through the AWS and GCP marketplaces and needs a Subscription ID and license bucket tied to registered AWS account IDs. No unit price is published on the pages read. |
| Ledger entries | [Gluten with Velox, TPC-DS 3 TB (project homepage)](/ledger/apache-gluten-velox-tpcds-3tb/) (run by its maker); [Gluten with Velox, TPC-H 3 TB (project homepage)](/ledger/apache-gluten-velox-tpch-3tb/) (run by its maker) | [Nucleus v3.0 against NVIDIA cuDF 25.04.00, operator tests](/ledger/datapelago-nucleus-vs-cudf-operators/) (run by its maker) |

Options in alphabetical order. Vendor statements are the vendor's own. Read 10 Nov 2026.

## How to read this page

The rows hold what each vendor or a named third party published, with no conclusion drawn. Benchmarks in the two columns were run on different hardware, data and baselines unless the Ledger entry says otherwise, so the numbers are not comparable across columns. The [Ledger method](/ledger/method/) explains how entries are recorded.

## Sources

- Apache Gluten with the Velox backend: gluten.apache.org: https://gluten.apache.org/ (read 10 Nov 2026)
- Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-st: https://raw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-started.md (read 10 Nov 2026)
- Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/velox-b: https://raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/velox-backend-limitations.md (read 10 Nov 2026)
- Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta: https://raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-started/Velox.md (read 10 Nov 2026)
- Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta: https://raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-started/VeloxIceberg.md (read 10 Nov 2026)
- Apache Gluten with the Velox backend: gluten.apache.org/downloads: https://gluten.apache.org/downloads/ (read 10 Nov 2026)
- Apache Gluten with the Velox backend: raw.githubusercontent.com/apache/gluten/main/tools/gluten-it/README.md: https://raw.githubusercontent.com/apache/gluten/main/tools/gluten-it/README.md (read 10 Nov 2026)
- DataPelago: datapelago.ai: https://www.datapelago.ai/ (read 10 Nov 2026)
- DataPelago: datapelago.ai/dpa: https://www.datapelago.ai/dpa (read 10 Nov 2026)
- DataPelago: datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html: https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html (read 10 Nov 2026)
- DataPelago: datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html: https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html (read 10 Nov 2026)
- DataPelago: datapelago.ai/technology: https://www.datapelago.ai/technology (read 10 Nov 2026)
- DataPelago: datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF: https://www.datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF (read 10 Nov 2026)
- Apache Gluten Powered By page: https://gluten.apache.org/poweredby/ (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://sparkaccelerationcompare.com/compare/apache-gluten-velox-vs-datapelago/
