Side by side, read 10 Nov 2026
DualBird and NVIDIA cuDF plugin for Apache Spark: the published facts in the same rows
DualBird profileNVIDIA cuDF plugin for Apache Spark profile
The same rows for both
| Row | DualBird | NVIDIA cuDF plugin for Apache Spark |
|---|---|---|
| Kind | FPGA Spark plugin on AWS F2 | GPU plugin, formerly RAPIDS Accelerator |
| Does well | DualBird publishes a worked example: a sort-based Iceberg compaction benchmark on 100 GB, with the dataset shape (about 370 million rows, 64 columns, ZSTD level 3) and two baselines (vanilla Spark and an unnamed C++ accelerated Spark). DualBird says onboarding is an instance type change to F2 plus its Spark plugin, and that Spark still handles planning, scheduling and fault tolerance. | AWS's August 2026 post publishes runtime and cost per run: 281 seconds and $2.06 on GPU versus 1,010 to 1,043 seconds and $2.93 to $3.18 on CPU clusters. Amazon EMR on EKS supports the plugin natively from release 6.9.0, and NVIDIA lists Databricks 14.3 ML LTS and 17.3 ML LTS and Dataproc 2.1 to 2.3 as supported. |
| Limited | DualBird runs on AWS F2 instances only, so it is not an option on Azure, Google Cloud or a Databricks-managed cluster, and every benchmark published for it so far is its own. F2 is listed in eight AWS regions (not, for example, us-east-2 or eu-west-1), the method for its compaction benchmark is behind a sign-up form, and no price for the plugin is published. A DualBird engineer states that workloads dominated by Python UDFs or unsupported operators fall back to the CPU and do not see the same benefit yet. | The plugin needs NVIDIA GPUs of the Volta generation or newer, so the cluster moves to GPU instances such as g7.4xlarge. Only the DataFrame API and a limited set of operations are accelerated; RDDs, GraphX and APIs that take Scala or Java functions are not. |
| Published speed results | The home page body text states the headline claim as 10x-30x faster data processing with 50%-90% lower costs and no changes required. The same wording appears in the page's JSON-LD organization description and in the April 28, 2026 blog post. The raw HTML of the home page has a different range in its meta description and og:description (and twitter:description): 10x-100x faster. This is the meta tag, not body text. The February 5, 2026 funding blog post also says 10-100x faster and 50-90% cost savings. The body text of the home and product pages says 10x-30x, so the two ranges differ by site location. The product page repeats the 10x-30x end-to-end figure labeled 'Instance for instance' and 50%-90% cost reduction labeled 'Workload for workload'. The meta description of the product page is only 'Plug-and-play Spark acceleration.' Home and product pages show a '110GB Enrichment Pipeline (JOIN Denormalization)' benchmark: 50%-85% lower EC2 cost with 6x-18x higher performance ceiling. The pages give no hardware, baseline, dates or method, and no link to a write-up was found. Sort-based Iceberg compaction on 100 GB, compared with vanilla Spark and with an unnamed 'C++ accelerated Spark' (unnamed) with tuned EBS, memory and parallelism at each cluster size. Dataset is synthetic: about 370 million rows, 64 mixed-type columns, ZSTD level 3. Results: roughly 12-20x faster single-task execution, 55-85% lower EC2 cost, stable cost across scaling, about 6x higher practical performance ceiling. Cost per terabyte compacted is stated as about $2/TB for DualBird versus about $6/TB rising past $13/TB for CPU clusters. Part of the saving is attributed to F2 local NVMe replacing EBS volumes. The home page shows 'X15-X20' single-task speedup for the same benchmark, while the June 11, 2026 blog and the white paper landing page say 12-20x. The two do not match. The 'Single Task Performance' home-page tile says X15-X20 speedup for the same cluster size while reducing cost by over 50%. The blog describes the single-task comparison as against vanilla Spark, and the April 28 post as against 'a high-end C++ columnar engine running on a single CPU core'. The baseline wording is therefore not consistent across pages. | AWS and NVIDIA report that TPC-DS at 3 TB (103 queries) on 8-node Amazon EMR on EKS clusters ran in 281 seconds on g7.4xlarge GPU instances (NVIDIA RTX PRO 4500 Blackwell) versus 1,010 to 1,043 seconds on comparable CPU instances, which AWS labels a 3.7x speedup. Median of 5 iterations, Spark 3.5.6, cuDF plugin 26.04.2, us-east-1. NVIDIA's own FAQ gives a general expectation and says there is no formula for a specific query. This is a vendor claim with no named benchmark behind it. NVIDIA's overview page compares the published AWS GPU result with separate NDS (TPC-DS derived) 3 TB runs of open-source Spark 3.5.6 on EC2. It says the GPU run is 8.4x faster than m7gd.8xlarge and 11.0x faster than m9gd.4xlarge, with 77% and 72% lower reported cost per run. NVIDIA itself notes the runs use different hardware and software stacks. |
| Published cost results and prices | No pricing, price list or marketplace listing was found on the site. Cost claims are only percentages (50%-90% lower, 55%-85% lower EC2) and the about $2/TB versus $6-13/TB compaction figure. The pricing model for the plugin itself is not stated, so users cannot tell whether F2 instance price or license fee is included in the claimed savings. | AWS reports cost per benchmark run of $2.06 on the GPU cluster versus $2.93 to $3.18 on the CPU clusters, using On-Demand us-east-1 prices as of May 2026. The g7.4xlarge cluster costs $26.35 per hour (8 x $3.042 EC2 plus $2.01 for EMR on EKS) against $10.43 to $10.99 per hour for the CPU clusters. Cost excludes the EKS control-plane fee, EBS, S3 and the driver pod. The software itself is free; you pay for GPU instances. |
| Code changes | Onboarding is two changes: switch the EC2 instance type to F2 and add the DualBird Spark plugin. The page shows three steps: change instance type, use our Spark plugin, see the results. The about page says 'No infrastructure changes. No code changes.' The compaction blog says Spark still handles planning, scheduling and fault tolerance and no pipeline rewrite or platform migration is needed. A DualBird engineer states the FPGA path accelerates a defined set of operators (sort, join, aggregation and others). Workloads dominated by Python UDFs or unsupported operators fall back to CPU and do not see the same benefit yet. Coverage is described as expanding. | NVIDIA says existing Spark applications run with no code change once the plugin is launched and enabled by a configuration setting. Unsupported operators and UDFs fall back to the CPU, so the job completes but those stages are not accelerated. Only the DataFrame API (Dataset[Row]) and a limited set of SQL and DataFrame operations are accelerated. Custom classes or types with Dataset, APIs that take Row or Scala/Java functions (flatMap, foreach, foreachPartition), GraphX and RDDs are not accelerated. Structured streaming is not a focus. |
| Tuning | DualBird claims tens of Spark settings (executor sizing, CPU-memory ratios, parallelism) become irrelevant, disk spills effectively disappear and data skew stops causing spills, because only a few tasks run at once on the FPGA. The product page says the majority of common Spark performance configurations do not exist when using the solution. Cluster sizing reduces to the number of F2 nodes. These are vendor claims, not independently tested. Part 2 of the series (Sep 1, 2026) says skew still costs time, but less, and that it does not need repartitioning or salting. | The user guide has a tuning guide (executors, tasks per executor, pooled and pinned memory, concurrent tasks per GPU, shuffle partitions, input partition size, columnar batch size), an AutoTuner, a Profiling tool and a Qualification tool, plus a recommended-practices section on handling GPU out-of-memory errors. In the AWS test the GPU config set executor memoryOverhead 30G, pinnedPool 8G, concurrentGpuTasks 3 and RapidsShuffleManager, and AWS advises using the Spark History Server to tune concurrentGpuTasks and pinnedPool.size. AWS recommends a pre-migration assessment with the NVIDIA qualification tool to estimate GPU acceleration potential, and setting spark.rapids.sql.explain=NOT_ON_GPU to see which operations run on CPU. |
| Spark versions, platforms and table formats | The home page says the plugin is compatible with Apache Spark, Amazon EMR and Amazon EKS. It lists 'Optimized out of the box', 'Migration-free' and 'No more performance tuning and troubleshooting'. No Spark version numbers, Databricks support or Iceberg/Delta/Hudi support list are stated on the home, product, about or resources pages. Iceberg appears only in the compaction benchmark and in the meta description of the Dec 1, 2025 demo post ('Accelerate Apache Spark and Apache Iceberg'). | Release v26.10.0 supports Apache Spark 3.3.x, 3.4.x, 3.5.0 to 3.5.9, 4.0.0 to 4.0.4, 4.1.1 to 4.1.3 and 4.2.0 (Scala 2.12 and 2.13 depending on version). Supported Databricks runtimes on AWS and Azure are 14.3 ML LTS (Spark 3.5.0) and 17.3 ML LTS (Spark 4.0.0). Dataproc 2.1 to 2.3 and Dataproc Serverless runtimes 1.2, 2.2 and 3.0 are supported. Amazon EMR on EKS natively supports the plugin (release 6.9.0 or later; the '-spark-rapids' release variant ships it pre-installed). The user guide also has a getting-started page for AWS EMR (EC2) and covers Apache Iceberg and Delta Lake support pages, including reading and writing tables. |
| Where it runs | DualBird needs AWS EC2 F2 instances. AWS describes F2 as second-generation FPGA instances with up to 8 AMD Virtex UltraScale+ HBM VU47P FPGAs, 16 GiB HBM per FPGA, 3rd-gen AMD EPYC (Milan) CPUs up to 192 vCPU, up to 2 TiB memory and up to 7.6 TiB NVMe. Sizes: f2.6xlarge (1 FPGA, 24 vCPU, 256 GiB), f2.12xlarge (2 FPGAs, 48 vCPU, 512 GiB), f2.48xlarge (8 FPGAs, 192 vCPU, 2,048 GiB). The AWS 'instance types by Region' doc lists F2 in eight regions: us-east-1 (N. Virginia), us-west-2 (Oregon), ap-northeast-1 (Tokyo), ap-northeast-2 (Seoul), ap-southeast-2 (Sydney), ca-central-1 (Canada Central), eu-central-1 (Frankfurt) and eu-west-2 (London). F2 is not listed in other regions on that page (for example us-east-2, eu-west-1). DualBird is AWS-only: no other cloud is mentioned. The product page says deployment runs on the customer's own servers in their AWS environment and 'Everything stays under your control'. | Needs NVIDIA GPUs: Volta, Turing, Ampere, Ada Lovelace, Hopper and Blackwell generations; tested on V100, T4, A10, A100, L4, H100, B100, RTX PRO 4500 and RTX PRO 6000. Linux with glibc 2.28 or newer; NVIDIA driver R525+ (CUDA 12) or R580+ (CUDA 13). On AWS the benchmark used g7.4xlarge (1 GPU, 16 vCPU, 64 GB RAM); NVIDIA's EMR guide maps m5d.8xlarge to g4dn.8xlarge for T4 GPUs. |
| How open its benchmarks are | The white paper (dated July 1, 2026) has a landing page that restates the results and says methodology, system architecture and detailed analysis are in a download. The full document is gated behind a form and was not readable. The blog gives dataset shape and baselines but not instance counts, Spark version, configs or scripts. No public repo was found (github.com/dualbird returns 404). | NVIDIA publishes the NDS benchmark automation (data generation, power and throughput runs) in a public Apache-2.0 repo, with a disclaimer that NDS is derived from TPC-DS and is not comparable to published TPC-DS results. The AWS post publishes the cluster, Spark config, EMR release and pricing method; it does not link scripts. |
| Change Budget: code | No code change stated. DualBird's about page says 'No infrastructure changes. No code changes.' | No code change stated. NVIDIA says existing Spark applications run with no code change once the plugin is launched and enabled by a configuration setting. Unsupported operators and UDFs fall back to the CPU. |
| Change Budget: cluster | Instance type change to F2. DualBird's onboarding is two changes: switch the EC2 instance type to F2 and add the DualBird Spark plugin. Cluster sizing then reduces to the number of F2 nodes, per DualBird. | GPU instances and plugin settings. The cluster needs NVIDIA GPUs (Volta to Blackwell generations) and a current driver. AWS's test used g7.4xlarge nodes with executor memoryOverhead 30G, pinnedPool 8G, concurrentGpuTasks 3 and RapidsShuffleManager. |
| Change Budget: cloud | AWS only, F2 in eight regions. F2 is listed in us-east-1, us-west-2, ap-northeast-1, ap-northeast-2, ap-southeast-2, ca-central-1, eu-central-1 and eu-west-2, and not, for example, in us-east-2 or eu-west-1. DualBird mentions no other cloud. | AWS, Azure and Google Cloud documented. NVIDIA lists Databricks runtimes on AWS and Azure and Dataproc on Google Cloud; Amazon EMR on EKS supports the plugin natively from release 6.9.0. |
| Change Budget: vendor | Plugin contract; no price published. No pricing, price list or marketplace listing was found on DualBird's site, and the pricing model for the plugin is not stated. | Free software; GPU instances paid. The software is free (source Apache-2.0); you pay for GPU instances, and AWS's cost per run includes EC2 and EMR on EKS charges. |
| Ledger entries | DualBird sort-based Iceberg compaction, 100 GB (run by its maker); DualBird 110 GB enrichment pipeline (join denormalization) (run by its maker); DualBird single task performance tile (run by its maker) | GPU Spark on Amazon EMR on EKS with G7 instances, TPC-DS 3 TB (run by its maker); NVIDIA comparison with open-source Spark on EC2, NDS 3 TB (run by its maker) |
How to read this page
The rows hold what each vendor or a named third party published, with no conclusion drawn. Benchmarks in the two columns were run on different hardware, data and baselines unless the Ledger entry says otherwise, so the numbers are not comparable across columns. The Ledger method explains how entries are recorded.
Sources
- DualBird: dualbird.io, read 10 Nov 2026
- DualBird: dualbird.io/product, read 10 Nov 2026
- DualBird: dualbird.io/resources/blog/a-simpler-spark, read 10 Nov 2026
- DualBird: dualbird.io/resources/blog/iceberg-compaction-benchmark-faster-spark-low, read 10 Nov 2026
- DualBird: dualbird.io/resources/blog/spark-iceberg-compaction-benchmark-whitepaper, read 10 Nov 2026
- DualBird: aws.amazon.com/ec2/instance-types/f2, read 10 Nov 2026
- DualBird: docs.aws.amazon.com/ec2/latest/instancetypes/ec2-instance-regions.html, read 10 Nov 2026
- NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): aws.amazon.com/blogs/big-data/gpu-accelerated-apache-spark-with-amazon-e, read 10 Nov 2026
- NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): docs.nvidia.com/spark-rapids/user-guide/latest/faq.html, read 10 Nov 2026
- NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): docs.nvidia.com/spark-rapids/user-guide/latest/overview.html, read 10 Nov 2026
- NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): docs.nvidia.com/spark-rapids/user-guide/latest/index.html, read 10 Nov 2026
- NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): nvidia.github.io/cudf-spark/docs/download.html, read 10 Nov 2026
- NVIDIA RAPIDS Accelerator for Apache Spark (now NVIDIA cuDF plugin for Apache Spark): raw.githubusercontent.com/NVIDIA/cudf-spark-benchmarks/dev/nds/README.md, read 10 Nov 2026
- DualBird about page, read 10 Nov 2026
Last reviewed 10 Nov 2026.