Skip to content
Spark Acceleration Compare

Ledger entry, published 2026-06-11

DualBird sort-based Iceberg compaction, 100 GB

All Ledger entriesDualBird profile

Option tested
DualBird
Ran by
DualBird (the maker of the option tested)
Published
2026-06-11
Workload
Sort-based Apache Iceberg compaction on a synthetic denormalized event table (about 370 million rows, 64 columns, ZSTD level 3)
Data size
100 GB
Hardware
AWS F2 instances for DualBird; baselines are vanilla Spark and an unnamed C++ accelerated Spark; instance types and counts not stated in the public post
Result as published
About 12x to 20x faster task execution against vanilla Spark and an unnamed C++ accelerated Spark (DualBird's wording; no absolute times published)
Cost result as published
55% to 85% lower EC2 cost; about $2 per TB for DualBird against about $6 per TB rising past $13 per TB for CPU-based clusters
Method published
Behind a sign-up form (white paper dated 2026-07-01); no public repository found

What the test shows and does not show

DualBird attributes part of the saving to F2 local NVMe replacing EBS volumes.

The post also uses the phrases 'higher practical performance ceiling' and 'stable compaction cost across compute scaling ranges'; both are DualBird's wording and neither is a measurement, so they are not in the result cell.

The result date (2026-06-11) is the date on the blog post. The white paper that holds the method is dated 2026-07-01, about three weeks later, so the method was released after the result.

The home page shows X15 to X20 for the same test (the 'DualBird single task performance tile' entry) while the post says 12x to 20x; instance counts, Spark version and scripts are not public.

Other Ledger entries for DualBird

Sources

  1. DualBird Iceberg compaction benchmark post, read 10 Nov 2026

Last reviewed 10 Nov 2026.