# DataPelago and Microsoft Fabric native execution engine: the published facts in the same rows

Side by side, read 10 Nov 2026

**The short answer.** DataPelago and Microsoft Fabric native execution engine are set out here in the same rows: published speed and cost results, code and cluster changes, tuning, stack fit, where each runs and how open the tests are. The Ledger holds 1 published entry for DataPelago and no entries for Microsoft Fabric native execution engine, each with who ran it.

## The same rows for both

| Row | DataPelago | Microsoft Fabric native execution engine |
| --- | --- | --- |
| Kind | Gluten-based CPU and GPU accelerator | Velox-based engine inside Fabric Spark |
| Does well | Per DataPelago's documentation, DPA-S runs in the customer's own cloud account on EKS, Dataproc 2.2.49 with Spark 3.5.1, Dataproc 2.1.75 with Spark 3.3.2, and GKE. DataPelago's blog publishes operator microbenchmarks of Nucleus v3.0 against NVIDIA cuDF 25.04.00 on one H100 GPU of an AWS p5.48xlarge. | Per Microsoft's documentation, no code modifications are required, and the engine is enabled with an Acceleration tab toggle or spark.native.enabled. The engine works with Parquet, Delta and CSV, including data reached through OneLake shortcuts. |
| Limited | DataPelago's 10X and 80% claims, and its customer quotes of 50% lower cost, carry no baseline, workload or method. Pricing is pay-as-you-go through the AWS and GCP marketplaces, and no unit price is published on the pages read. The AWS docs describe EKS with Spark 3.5.1 only, and EMR and Databricks are not mentioned. | The engine runs only inside Microsoft Fabric on Azure and does not run on AWS EMR or EKS. Microsoft's page gives no number, workload or dataset for the speedup, and a benchmark method or scripts: none found on 2026-11-10. |
| Published speed results | The home page headline claims 10X faster at 80% cost reduction for GenAI and analytics data processing. The DPA-S page says it can reduce costs by up to 80% on qualifying workloads without rewriting applications or migrating data. No workload, baseline or method is attached to the 10X/80% figures. DataPelago's blog (Aug 21, 2025) compares Nucleus v3.0 with NVIDIA cuDF 25.04.00 on one H100 of an AWS p5.48xlarge, for single operators. Nucleus is up to 10.5x faster on project, up to 10.1x on filter, up to 4.3x on aggregate, up to 4.5x on hash aggregate, and up to 8.2x on Top-K. These are operator microbenchmarks against a GPU library, not end-to-end Spark jobs. | The page says the engine significantly raises query performance while minimizing operational costs, but gives no number, workload or dataset. 'Actual results vary by workload characteristics and configuration.' |
| Published cost results and prices | Named customer quotes on the vendor site: Akad Seguros (CTO) says costs fell by more than 50%; ShareChat (VP of Data) says the heavy OLAP cube jobs now complete on OSS Spark and costs fell by 50%; RevSure.ai (CEO) reports gains in days with zero application changes; McAfee and Samsung SDS America describe 'promising results' only. These are testimonials with no baseline, workload or method. The DPA-S page describes pay-as-you-go pricing billed through the AWS and GCP marketplaces ('Use more and Save more'). No price per unit is published on the pages read. The docs require a Subscription ID and license bucket path tied to registered AWS account IDs. | Fabric is billed by capacity units (F2 to F4096 SKUs), pay-as-you-go or reservation. An opt-in Autoscale Billing for Spark runs Spark jobs on dedicated serverless resources at standard regional pay-as-you-go CU-hour rates, with a base Fabric capacity still required. SKU prices on the page load dynamically and were not readable. |
| Code changes | DPA-S is presented as working with existing Spark clusters and jobs with no application rewrites or data migration; languages are PySpark, Scala and Spark SQL. Workloads to select: compute-intensive Parquet/ORC SQL and DataFrame jobs. Workloads to avoid: heavy use of RDDs and UDFs. The docs list limitations: native CPU acceleration is not used when ANSI mode is on (queries run on the JVM), case-sensitive mode is not supported and may return incorrect results, plus listed caveats for regexp functions, file source formats, JSON functions and NaN. | Docs say no code modifications are required and no vendor lock-in. Unsupported operators fall back to JVM Spark. Listed limitations include decimal-to-float casting, timezone, round(), map() duplicate-key and collect_list ordering differences on Runtime 1.3, said to be resolved in Runtime 2.0. |
| Tuning | Mandatory Spark settings in the EKS docs include spark.plugins=org.apache.gluten.GlutenPlugin, the subscription ID and license location env vars, spark.memory.offHeap.enabled=true with a positive spark.memory.offHeap.size, spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager and netty Java options. CBO settings are recommended. The DPA-S page tells users to adjust Spark configuration (shuffle manager, memory tuning). So some per-deployment configuration is required, in contrast to 'no tuning' claims by other vendors. | Enable via the Acceleration tab toggle in an environment, or set spark.native.enabled to true at the start of a notebook or Spark job definition. Per-query disable is by setting the flag to false and re-enabling in later cells. Fabric Spark Advisor alerts show fallback. |
| Spark versions, platforms and table formats | On AWS, DPA-S is available for OSS Spark on EKS as a Docker container in ECR: Spark 3.5.1, Java 11, Scala 2.12, Ubuntu 22. EMR and Databricks are not mentioned on the AWS docs page. The vendor page says 'multiple Spark versions on Google Cloud and AWS'. Lakehouse formats Iceberg, Hudi and Delta are named only in a reseller testimonial (Twingo) on the vendor site. GCP docs list supported combinations: Dataproc 2.2.49 with Spark 3.5.1, Dataproc 2.1.75 with Spark 3.3.2, OSS Spark 3.5.1 and OSS Spark 3.3.4 (all Java 11, Scala 2.12) and a GKE deployment. Docs version header is 3.1.7. The technology page says the engine builds on Substrait-based open-source frameworks such as Apache Gluten and Velox; the docs configure the Gluten plugin class. The Gluten initiator is quoted on the home page in support. Nucleus and DPA-S themselves are proprietary and licensed. | Supports Runtime 1.3 (Spark 3.5, Delta Lake 3.2) and Runtime 2.0 (Spark 4.1, Delta Lake 4.1). Works with Parquet, Delta and CSV, including data reached via OneLake shortcuts. Not an AWS service. |
| Where it runs | Runs in the customer's own cloud account on AWS (EKS) and Google Cloud (Dataproc, GKE). The Nucleus engine targets CPU, GPU and FPGA. The Nucleus-versus-cuDF test used an AWS p5.48xlarge with H100 GPUs; the DPA-S docs pages read do not state a GPU requirement, so the hardware needed for DPA-S itself was not confirmed. | Runs only inside Microsoft Fabric (Azure) on OneLake data. It does not run on AWS EMR or EKS. |
| How open its benchmarks are | None found on 10 Nov 2026. | No benchmark method, data or scripts are published for the engine's speedup on the documentation page. |
| Change Budget: code | No application rewrite stated. DataPelago presents DPA-S as working with existing Spark clusters and jobs with no application rewrites or data migration. It tells users to avoid heavy use of RDDs and UDFs. | No code modifications stated. Microsoft's documentation says no code modifications are required. Unsupported operators fall back to JVM Spark. |
| Change Budget: cluster | Plugin, license and memory settings. Mandatory settings include the Gluten plugin class, the subscription ID and license location, off-heap memory, the columnar shuffle manager and Netty Java options. On AWS it ships as a Docker container in ECR for Spark 3.5.1 on EKS. | Toggle in environment or Spark config. Enable it with the Acceleration tab toggle in an environment, or set spark.native.enabled to true in a notebook or Spark job definition. It needs Runtime 1.3 or 2.0. |
| Change Budget: cloud | AWS EKS or Google Cloud. DataPelago documents EKS on AWS and Dataproc and GKE on Google Cloud, in the customer's own account. | Microsoft Fabric on Azure only. The engine runs only inside Microsoft Fabric on OneLake data, and does not run on AWS EMR or EKS. |
| Change Budget: vendor | Marketplace subscription required. DPA-S is billed pay-as-you-go through the AWS and GCP marketplaces and needs a Subscription ID and license bucket tied to registered AWS account IDs. No unit price is published on the pages read. | Fabric capacity required. Fabric is billed by capacity units (F2 to F4096 SKUs), pay-as-you-go or reservation. Autoscale Billing for Spark still requires a base Fabric capacity. SKU prices were not readable on the page. |
| Ledger entries | [Nucleus v3.0 against NVIDIA cuDF 25.04.00, operator tests](/ledger/datapelago-nucleus-vs-cudf-operators/) (run by its maker) | None found on 10 Nov 2026. |

Options in alphabetical order. Vendor statements are the vendor's own. Read 10 Nov 2026.

## How to read this page

The rows hold what each vendor or a named third party published, with no conclusion drawn. Benchmarks in the two columns were run on different hardware, data and baselines unless the Ledger entry says otherwise, so the numbers are not comparable across columns. The [Ledger method](/ledger/method/) explains how entries are recorded.

## Sources

- DataPelago: datapelago.ai: https://www.datapelago.ai/ (read 10 Nov 2026)
- DataPelago: datapelago.ai/dpa: https://www.datapelago.ai/dpa (read 10 Nov 2026)
- DataPelago: datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html: https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-aws.html (read 10 Nov 2026)
- DataPelago: datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html: https://datapelago-docs.readthedocs-hosted.com/en/latest/deploy-on-gcp.html (read 10 Nov 2026)
- DataPelago: datapelago.ai/technology: https://www.datapelago.ai/technology (read 10 Nov 2026)
- DataPelago: datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF: https://www.datapelago.ai/resources/DataPelago-Nucleus-Vs-Nvidia-cuDF (read 10 Nov 2026)
- Microsoft Fabric Spark native execution engine: learn.microsoft.com/en-us/fabric/data-engineering/native-execution-engin: https://learn.microsoft.com/en-us/fabric/data-engineering/native-execution-engine-overview (read 10 Nov 2026)
- Microsoft Fabric Spark native execution engine: azure.microsoft.com/en-us/pricing/details/microsoft-fabric: https://azure.microsoft.com/en-us/pricing/details/microsoft-fabric/ (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://sparkaccelerationcompare.com/compare/datapelago-vs-fabric-native-execution-engine/
