# Spill

Glossary, read 10 Nov 2026

**The short answer.** Spill is what happens when Spark cannot hold shuffle or aggregation data in execution memory and writes it to local disk instead. The Spark web UI reports two figures per stage: shuffle spill (memory), the deserialized size in RAM, and shuffle spill (disk), the serialized size written out.

Spill is a symptom, not a setting. It usually points to partitions that are too large, skewed keys, or executors with too little memory per task. Fixing it by hand means changing partition counts and memory ratios.

Some vendors say their option reduces or removes spill. Those are vendor claims; read them against the Spark UI numbers from a real job.

## A sourced example

The Spark configuration reference lists spark.shuffle.spill.diskWriteBufferSize, default 1024 * 1024 bytes (1 MiB), the buffer used when writing sorted records out to the on-disk spill file.

## Sources

- Apache Spark documentation: Web UI, Stages tab: https://spark.apache.org/docs/latest/web-ui.html (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://sparkaccelerationcompare.com/glossary/spill/
