# DataPelago, Gluten-based CPU and GPU accelerator: the published facts

Option profile, read 10 Nov 2026

**The short answer.** DataPelago Accelerator for Spark (DPA-S), launched in August 2025, is a proprietary Gluten-based plugin on the company's Nucleus engine. Per its documentation it runs on EKS in AWS and on Dataproc and GKE in Google Cloud. DataPelago claims 10X faster at 80% lower cost, with no workload or method attached.

- **Kind:** Gluten-based CPU and GPU accelerator
- **Website:** [www.datapelago.ai](https://www.datapelago.ai/)
- **Name checked:** DataPelago, Inc. (Mountain View, CA) is live. datapelago.com redirects to www.datapelago.ai; page links also reference datapelago.io. The Spark product is named DataPelago Accelerator for Spark (DPA-S), launched Aug 2025.
- **Read:** 10 Nov 2026

## What it does well

- Per DataPelago's documentation, DPA-S runs in the customer's own cloud account on EKS, Dataproc 2.2.49 with Spark 3.5.1, Dataproc 2.1.75 with Spark 3.3.2, and GKE.
- DataPelago's blog publishes operator microbenchmarks of Nucleus v3.0 against NVIDIA cuDF 25.04.00 on one H100 GPU of an AWS p5.48xlarge.

## Where it is limited

- DataPelago's 10X and 80% claims, and its customer quotes of 50% lower cost, carry no baseline, workload or method.
- Pricing is pay-as-you-go through the AWS and GCP marketplaces, and no unit price is published on the pages read.
- The AWS docs describe EKS with Spark 3.5.1 only, and EMR and Databricks are not mentioned.

## Published speed results

| Fact | Source type | Source, read |
| --- | --- | --- |
| The home page headline claims 10X faster at 80% cost reduction for GenAI and analytics data processing. The DPA-S page says it can reduce costs by up to 80% on qualifying workloads without rewriting applications or migrating data. No workload, baseline or method is attached to the 10X/80% figures. | vendor | [datapelago.ai](https://www.datapelago.ai/), 10 Nov 2026 |
| DataPelago's blog (Aug 21, 2025) compares Nucleus v3.0 with NVIDIA cuDF 25.04.00 on one H100 of an AWS p5.48xlarge, for single operators. Nucleus is up to 10.5x faster on project, up to 10.1x on filter, up to 4.3x on aggregate, up to 4.5x on hash aggregate, and up to 8.2x on Top-K. These are operator microbenchmarks against a GPU library, not end-to-end Spark jobs. | vendor | [datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF](https://www.datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF), 10 Nov 2026 |

## Published cost results and prices

| Fact | Source type | Source, read |
| --- | --- | --- |
| Named customer quotes on the vendor site: Akad Seguros (CTO) says costs fell by more than 50%; ShareChat (VP of Data) says the heavy OLAP cube jobs now complete on OSS Spark and costs fell by 50%; RevSure.ai (CEO) reports gains in days with zero application changes; McAfee and Samsung SDS America describe 'promising results' only. These are testimonials with no baseline, workload or method. | vendor | [datapelago.ai](https://www.datapelago.ai/), 10 Nov 2026 |
| The DPA-S page describes pay-as-you-go pricing billed through the AWS and GCP marketplaces ('Use more and Save more'). No price per unit is published on the pages read. The docs require a Subscription ID and license bucket path tied to registered AWS account IDs. | vendor | [datapelago.ai/dpa](https://www.datapelago.ai/dpa), 10 Nov 2026 |

## Code changes

| Fact | Source type | Source, read |
| --- | --- | --- |
| DPA-S is presented as working with existing Spark clusters and jobs with no application rewrites or data migration; languages are PySpark, Scala and Spark SQL. Workloads to select: compute-intensive Parquet/ORC SQL and DataFrame jobs. Workloads to avoid: heavy use of RDDs and UDFs. The docs list limitations: native CPU acceleration is not used when ANSI mode is on (queries run on the JVM), case-sensitive mode is not supported and may return incorrect results, plus listed caveats for regexp functions, file source formats, JSON functions and NaN. | vendor | [datapelago.ai/dpa](https://www.datapelago.ai/dpa), 10 Nov 2026 |

## Tuning

| Fact | Source type | Source, read |
| --- | --- | --- |
| Mandatory Spark settings in the EKS docs include spark.plugins=org.apache.gluten.GlutenPlugin, the subscription ID and license location env vars, spark.memory.offHeap.enabled=true with a positive spark.memory.offHeap.size, spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager and netty Java options. CBO settings are recommended. The DPA-S page tells users to adjust Spark configuration (shuffle manager, memory tuning). So some per-deployment configuration is required, in contrast to 'no tuning' claims by other vendors. | vendor docs | [datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html](https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html), 10 Nov 2026 |

## Spark versions, platforms and table formats

| Fact | Source type | Source, read |
| --- | --- | --- |
| On AWS, DPA-S is available for OSS Spark on EKS as a Docker container in ECR: Spark 3.5.1, Java 11, Scala 2.12, Ubuntu 22. EMR and Databricks are not mentioned on the AWS docs page. The vendor page says 'multiple Spark versions on Google Cloud and AWS'. Lakehouse formats Iceberg, Hudi and Delta are named only in a reseller testimonial (Twingo) on the vendor site. | vendor docs | [datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html](https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html), 10 Nov 2026 |
| GCP docs list supported combinations: Dataproc 2.2.49 with Spark 3.5.1, Dataproc 2.1.75 with Spark 3.3.2, OSS Spark 3.5.1 and OSS Spark 3.3.4 (all Java 11, Scala 2.12) and a GKE deployment. Docs version header is 3.1.7. | vendor docs | [datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html](https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html), 10 Nov 2026 |
| The technology page says the engine builds on Substrait-based open-source frameworks such as Apache Gluten and Velox; the docs configure the Gluten plugin class. The Gluten initiator is quoted on the home page in support. Nucleus and DPA-S themselves are proprietary and licensed. | vendor | [datapelago.ai](https://www.datapelago.ai/), 10 Nov 2026 |

## Where it runs

| Fact | Source type | Source, read |
| --- | --- | --- |
| Runs in the customer's own cloud account on AWS (EKS) and Google Cloud (Dataproc, GKE). The Nucleus engine targets CPU, GPU and FPGA. The Nucleus-versus-cuDF test used an AWS p5.48xlarge with H100 GPUs; the DPA-S docs pages read do not state a GPU requirement, so the hardware needed for DPA-S itself was not confirmed. | vendor | [datapelago.ai/technology](https://www.datapelago.ai/technology), 10 Nov 2026 |

## Profile

| Fact | Source type | Source, read |
| --- | --- | --- |
| Founder and CEO Rajan Goyal was previously CTO at Fungible. DataPelago, Inc. is at 100 View Street, Mountain View, CA. Timeline from the press page: Universal Data Processing Engine unveiled Oct 1, 2024; President John Chirapurath appointed Jul 28, 2025; DPA-S launched Aug 5, 2025; Nucleus-versus-cuDF release Aug 22, 2025; Fast Company Most Innovative Companies of 2026 (Mar 24, 2026); Data Breakthrough award (May 13, 2026). Investor/partner logos on the home page include Taiwania, Qualcomm, Eclipse, Alter and Nautilus (alt text); no round sizes were found. | vendor | [datapelago.ai/press](https://www.datapelago.ai/press), 10 Nov 2026 |

## Ledger entries for DataPelago

| Published | Option | Test | Result as published | Ran by | Method published |
| --- | --- | --- | --- | --- | --- |
| 2025-08-21 | [DataPelago](/options/datapelago/) | [Nucleus v3.0 against NVIDIA cuDF 25.04.00, operator tests](/ledger/datapelago-nucleus-vs-cudf-operators/) | Up to 10.5x on project, 10.1x on filter, 4.3x on aggregate, 4.5x on hash aggregate and 8.2x on K-largest selection, against cuDF | DataPelago (its maker) | No |

## Change Budget row

| Column | What changes | Detail | Source, read |
| --- | --- | --- | --- |
| Code | No application rewrite stated | DataPelago presents DPA-S as working with existing Spark clusters and jobs with no application rewrites or data migration. It tells users to avoid heavy use of RDDs and UDFs. | [DataPelago Accelerator for Spark page](https://www.datapelago.ai/dpa), 10 Nov 2026 |
| Cluster | Plugin, license and memory settings | Mandatory settings include the Gluten plugin class, the subscription ID and license location, off-heap memory, the columnar shuffle manager and Netty Java options. On AWS it ships as a Docker container in ECR for Spark 3.5.1 on EKS. | [DataPelago AWS deployment docs](https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html), 10 Nov 2026 |
| Cloud | AWS EKS or Google Cloud | DataPelago documents EKS on AWS and Dataproc and GKE on Google Cloud, in the customer's own account. | [DataPelago Google Cloud deployment docs](https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html), 10 Nov 2026 |
| Vendor | Marketplace subscription required | DPA-S is billed pay-as-you-go through the AWS and GCP marketplaces and needs a Subscription ID and license bucket tied to registered AWS account IDs. No unit price is published on the pages read. | [DataPelago AWS deployment docs](https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html), 10 Nov 2026 |

## Searched for and none found on 10 Nov 2026

- TPC-DS/TPC-H or end-to-end Spark benchmark with published method, and any third-party benchmark (searched: datapelago.ai, datapelago.ai/dpa, datapelago.ai/technology)
- Funding amounts, round dates and lead investors (searched: datapelago.ai/company, datapelago.ai/press, datapelago.ai)
- Published unit prices (searched: datapelago.ai/dpa, datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html)

## DataPelago side by side with each other option

- [DataPelago and Amazon EMR runtime for Apache Spark](/compare/datapelago-vs-emr-spark-runtime/)
- [DataPelago and Apache Auron](/compare/apache-auron-vs-datapelago/)
- [DataPelago and Apache DataFusion Comet](/compare/datafusion-comet-vs-datapelago/)
- [DataPelago and Apache Gluten with Velox](/compare/apache-gluten-velox-vs-datapelago/)
- [DataPelago and Databricks Photon](/compare/databricks-photon-vs-datapelago/)
- [DataPelago and DualBird](/compare/datapelago-vs-dualbird/)
- [DataPelago and Flarion](/compare/datapelago-vs-flarion/)
- [DataPelago and Google Cloud Lightning Engine](/compare/datapelago-vs-dataproc-lightning-engine/)
- [DataPelago and Microsoft Fabric native execution engine](/compare/datapelago-vs-fabric-native-execution-engine/)
- [DataPelago and NVIDIA cuDF plugin for Apache Spark](/compare/datapelago-vs-nvidia-rapids-spark/)
- [DataPelago and Speedata](/compare/datapelago-vs-speedata/)

## Sources

- DataPelago: datapelago.ai: https://www.datapelago.ai/ (read 10 Nov 2026)
- DataPelago: datapelago.ai/dpa: https://www.datapelago.ai/dpa (read 10 Nov 2026)
- DataPelago: datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html: https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html (read 10 Nov 2026)
- DataPelago: datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html: https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html (read 10 Nov 2026)
- DataPelago: datapelago.ai/technology: https://www.datapelago.ai/technology (read 10 Nov 2026)
- DataPelago: datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF: https://www.datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF (read 10 Nov 2026)
- DataPelago: datapelago.ai/press: https://www.datapelago.ai/press (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://sparkaccelerationcompare.com/options/datapelago/
