Skip to content
Spark Acceleration Compare

Glossary, read 10 Nov 2026

Whole-stage code generation

All termsOpen the Ledger

Before it, Spark 1.x used the Volcano iterator model, where each operator returned one row per call. Fusing the operators removed most of that overhead for filters, aggregates and hash joins.

Native engines such as Velox take the idea further by processing batches of columns with vectorized code. See Tungsten and vectorized execution.

A sourced example

The 2016 post measured single-thread cost per row in Spark 1.6 against 2.0: filter 15 ns to 1.1 ns, sum without group 14 ns to 0.9 ns, Parquet decoding of one int column 120 ns to 13 ns.

Sources

  1. Databricks blog: Apache Spark as a Compiler (May 23, 2016), read 10 Nov 2026

Last reviewed 10 Nov 2026.