Skip to content
Spark Acceleration Compare

Glossary, read 10 Nov 2026

Iceberg compaction (rewrite_data_files)

All termsOpen the Ledger

Compaction is a Spark job you pay for, repeatedly. Streaming and frequent small writes make it a daily cost, so it is a common target for acceleration.

Sort-based compaction shuffles and sorts data, which is the work FPGA and native options say they speed up.

A sourced example

The Iceberg procedure defaults to a 512 MB target (write.target-file-size-bytes, 536870912), rewrites files under 75% of that size and files over 180% of it.

Sources

  1. Apache Iceberg documentation: Spark procedures, read 10 Nov 2026

Last reviewed 10 Nov 2026.