Glossary, read 10 Nov 2026
Iceberg compaction (rewrite_data_files)
Compaction is a Spark job you pay for, repeatedly. Streaming and frequent small writes make it a daily cost, so it is a common target for acceleration.
Sort-based compaction shuffles and sorts data, which is the work FPGA and native options say they speed up.
A sourced example
The Iceberg procedure defaults to a 512 MB target (write.target-file-size-bytes, 536870912), rewrites files under 75% of that size and files over 180% of it.
Sources
- Apache Iceberg documentation: Spark procedures, read 10 Nov 2026
Last reviewed 10 Nov 2026.