Skip to content
Spark Acceleration Compare

News, 23 Sep 2026

Google Lightning Engine reaches Dataproc 3.0 cluster images from 3.0.2

All newsOpen the Ledger

Under September 23, 2026, the Google Cloud release notes announce that Lightning Engine is supported in Managed Service for Apache Spark cluster image 3.0 starting with subminor 3.0.2, in Debian 13 and Ubuntu 24 variants.

The new serverless runtime versions are 1.2.89, 2.2.89 and 2.3.42. In the 2.3 runtime, Serverless batches with Lightning Engine use Apache Iceberg 1.10 by default, and Parquet footer caching is enabled by default for Lightning Engine (Velox). Google says that if executor out-of-memory or task failures spike on tiny files, setting spark.gluten.sql.columnar.backend.velox.cacheParquetFooters=false mitigates it.

The notes give no benchmark for the 3.0 images. An older entry on the same page, dated October 22, 2025, quotes Google's own figure of up to 4.3x faster than open-source Spark on TPC-H-like benchmarks, and says it requires the Premium tier on Serverless.

Sources

  1. Google Cloud Dataproc release notes, read 10 Nov 2026

Last reviewed 10 Nov 2026.