Glossary, read 10 Nov 2026
Off-heap memory
Off-heap use is why native plugins ask for new memory settings. The total container size still has to fit your executor limit, so the JVM heap usually shrinks when you grow the off-heap pool.
The Spark documentation notes the setting has no effect on heap usage, so if executors must fit a hard limit you shrink the heap yourself.
A sourced example
Comet's TPC-DS benchmark configuration used spark.memory.offHeap.size=32G with 64G executor memory, 10G overhead and spark.comet.exec.memoryPool.fraction=0.8.
Sources
- Apache Spark configuration reference: memory, read 10 Nov 2026
Last reviewed 10 Nov 2026.