News, read 10 Nov 2026
Spark acceleration news, dated and sourced
Open the LedgerCompare two options
15 items, newest first
- NVIDIA cuDF plugin for Apache Spark 26.10.0 released, adds Spark 4.2.0 support5 Nov 2026. NVIDIA published release v26.10.0 of its cuDF plugin for Apache Spark on November 5, 2026.
- Apache DataFusion Comet 1.2.0 released: 399 commits from 40 contributors3 Nov 2026. Apache DataFusion Comet 1.2.0 was published on November 3, 2026, four weeks after 1.1.0.
- Apache Spark 4.3.0 released with stateful Real-Time Mode and SCD Type 230 Oct 2026. Apache Spark 4.3.0 was announced on October 30, 2026, the fourth release of the 4.x line.
- Apache Spark 4.2.1, 4.1.4 and 4.0.5 maintenance releases announced27 Oct 2026. Apache Spark announced three maintenance releases on October 27, 2026: 4.2.1, 4.1.4 and 4.0.5.
- Amazon EMR on EKS cuts the EMR charge by 30 percent23 Oct 2026. AWS announced on October 23, 2026 a 30 percent reduction in the EMR charge added to EKS when running EMR on EKS.
- Apache DataFusion Comet 1.1.0 released: 416 commits from 41 contributors6 Oct 2026. Apache DataFusion Comet 1.1.0 was published on October 6, 2026.
- EMR Serverless raises the Spark shuffle limit from 200 GB to 1 TB per job1 Oct 2026. AWS announced on October 1, 2026 that Amazon EMR Serverless supports up to 1 TB of shuffle per job, raising the previous 200 GB limit.
- Amazon EMR on EKS adds interactive Spark sessions with Spark Connect24 Sep 2026. AWS announced on September 24, 2026 that Amazon EMR on EKS supports interactive Apache Spark sessions through Spark Connect.
- Google Lightning Engine reaches Dataproc 3.0 cluster images from 3.0.223 Sep 2026. Google's release notes dated September 23, 2026 say Lightning Engine is supported in Dataproc 3.0 cluster images from subminor version 3.0.2 (3.0.2-debian13 and 3.0.2-ubuntu24).
- Amazon EMR introduces Long Term Support, starting with emr-spark-8.1.0 on Spark 4.122 Sep 2026. Amazon EMR announced Long Term Support releases on September 22, 2026, starting with emr-spark-8.1.0 and Apache Spark 4.1.
- Amazon EMR 7.14 is available with Spark 3.5.8 and Iceberg 1.10.122 Sep 2026. AWS announced Amazon EMR 7.14 on September 22, 2026.
- Google Lightning Engine runs on Arm with Axion C4A for Dataproc 2.3 clusters1 Sep 2026. Google's Dataproc release notes of September 1, 2026 say Lightning Engine with Native Query Execution is supported on Arm for image version 2.3 clusters (2.3-ubuntu22-arm).
- DualBird blog Part 2: why data skew costs less on its FPGA architecture1 Sep 2026. DualBird published Part 2 of its Faster Spark series on September 1, 2026, written by its chief architect and co-founder.
- Apache Gluten 1.7.0 released with Spark 4.x test coverage and Delta 4 native write26 Aug 2026. Apache Gluten published release 1.7.0 on August 26, 2026, after three release candidates dated August 10, 17 and 21.
- Flarion blog: the hard part of a Spark upgrade is proving the output did not change18 Aug 2026. Flarion published a blog post on August 18, 2026 arguing that validation, not code rewriting, is the main obstacle to Spark version upgrades.
Last reviewed 10 Nov 2026.