Skip to content
Spark Acceleration Compare

Option profile, read 10 Nov 2026

Apache Gluten with Velox, Open-source Spark plugin offloading to Velox: the published facts

Apache Gluten with Velox in the LedgerCompare with another option

Kind
Open-source Spark plugin offloading to Velox
Website
gluten.apache.org
Name checked
The project is now called Apache Gluten and its homepage describes it as an Apache Top-Level Project. The GitHub repo apache/incubator-gluten now resolves to apache/gluten. Velox is Meta's C++ library; a Gluten release note says the Velox fork reference moved from oap-project to IBM.
Read
10 Nov 2026

What it does well

  • Gluten needs no DataFrame or SQL changes, only configuration: the GlutenPlugin class, off-heap memory and the columnar shuffle manager.
  • The repository includes the gluten-it tool for data generation, query runs and result comparison on Spark 3.4, 3.5, 4.0 and 4.1 profiles.

Where it is limited

  • The docs name Spark 3.4.4, 3.5.5, 4.0.2 and 4.1.1 as officially supported and warn that an unofficial Spark version can throw NoSuchMethodError at runtime.
  • Per the Velox backend docs, Iceberg support is read-only, Hudi support covers copy-on-write tables only, and ANSI mode always falls back to vanilla Spark.

Published speed results

FactSource typeSource, read
The project homepage reports TPC-H and TPC-DS at 3 TB on Intel Xeon Platinum 8592+ with Spark 3.3.1: overall 3.34x on TPC-H (up to 23.45x on single queries) and 3.02x on TPC-DS (up to 13.75x on single queries). The page does not link a method or scripts for these numbers; the hardware is from Intel, a contributor.open source projectgluten.apache.org, 10 Nov 2026

Code changes

FactSource typeSource, read
Gluten is a Spark plugin that needs no DataFrame or SQL changes, only configuration: spark.plugins=org.apache.gluten.GlutenPlugin, off-heap memory enabled with a size, ColumnarShuffleManager and the bundle jar.open source projectraw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-st, 10 Nov 2026
Unsupported operators fall back to vanilla Spark through columnar-to-row and row-to-columnar conversions. The Velox backend limitations page lists known fallbacks and differences: ANSI mode always falls back, case-sensitive mode may give wrong results, regexp functions use RE2 (no lookaround), file scans fully support only Parquet and partly ORC, partitioned scans need partition info in the path, and JSON functions differ for single-quoted strings.open source projectraw.githubusercontent.com/apache/gluten/main/docs/backends/velox/velox-b, 10 Nov 2026

Tuning

FactSource typeSource, read
Off-heap memory must be enabled and sized (the quick start uses spark.memory.offHeap.size=20g and says to adjust it for the environment), and the columnar shuffle manager is needed for native shuffle. Docs also describe a troubleshooting page, Celeborn and Uniffle remote shuffle settings, and a setting (spark.gluten.sql.fallbackRegexpExpressions) for users who cannot tolerate regexp differences. Binary jars must load ahead of Spark's jars because Gluten overrides some Spark classes.open source projectraw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-st, 10 Nov 2026

Spark versions, platforms and table formats

FactSource typeSource, read
The docs state officially supported Spark versions are 3.4.4, 3.5.5, 4.0.2 and 4.1.1 (Spark 4.x needs JDK 17+ and Scala 2.13). The downloads page still lists Spark 3.3 binaries for 1.6.0 and 1.7.0, and the homepage benchmark used Spark 3.3.1. Using an unofficial Spark version can throw NoSuchMethodError at runtime. Docs do not mention EMR, EKS or Databricks support.open source projectraw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta, 10 Nov 2026
Velox backend docs: Iceberg reads of copy-on-write and merge-on-read tables are supported but the Iceberg page says only reading is supported and only Spark 3.4 is well tested; Hudi supports reading COW tables only; Paimon supports non-primary-key tables only; Delta has a build profile and a doc page. Celeborn 0.3 to 0.5 and Uniffle 0.9.2 remote shuffle are supported.open source projectraw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta, 10 Nov 2026

Where it runs

FactSource typeSource, read
Official build and test platform is Ubuntu 20.04/22.04 or CentOS 8; the downloads page says binaries were built on CentOS 7 with static linking on Intel Xeon Platinum 8272CL class CPUs, and that performance and compatibility are not guaranteed on other OSes or hardware, so building from source is recommended. Docs have pages for S3, GCS and ABFS storage.open source projectgluten.apache.org/downloads, 10 Nov 2026

How open its benchmarks are

FactSource typeSource, read
The repo includes the gluten-it integration test and benchmark tool (data generation, query runs, result comparison against a baseline) for building and running TPC-style queries locally, supported on Spark 3.4, 3.5, 4.0 and 4.1 profiles. The homepage TPC-H and TPC-DS numbers do not themselves link to a run configuration.open source projectraw.githubusercontent.com/apache/gluten/main/tools/gluten-it/README.md, 10 Nov 2026

Company and breadth

FactSource typeSource, read
The Powered By page lists adopters including Alibaba, Baidu, Bytedance, BIGO, IBM, Kyligence, Meituan, Microsoft (Fabric), Google (Lightning engine), Netease, Oracle (OCI), Palantir, Pinterest, SF Express and Tencent. These are self-reported entries. The homepage benchmark runs on Intel hardware; the docs describe Velox as Meta's C++ execution library. Project license is Apache-2.0 (GitHub API).open source projectgluten.apache.org/poweredby, 10 Nov 2026

Profile

FactSource typeSource, read
GitHub releases: v1.7.0 published 2026-08-26 (rc1 2026-08-17). The downloads page lists 1.7.0 as 'Jun. 6 2025', which conflicts with the GitHub date and looks like a typo; 1.6.0 is listed as Mar. 10 2026 and 1.5.0 as Oct. 29 2025. Repo created 2021-12-06, about 1,600 stars.open source projectapi.github.com/repos/apache/incubator-gluten/releases?per_page=3, 10 Nov 2026

Ledger entries for Apache Gluten with Velox

PublishedOptionTestResult as publishedRan byMethod published
Not statedApache Gluten with VeloxGluten with Velox, TPC-DS 3 TB (project homepage)Gluten project's wording: overall 3.02x speedup with Gluten and Velox; up to 13.75x on individual queriesApache Gluten project (its maker)No
Not statedApache Gluten with VeloxGluten with Velox, TPC-H 3 TB (project homepage)Gluten project's wording: overall 3.34x speedup with Gluten and Velox; up to 23.45x on individual queriesApache Gluten project (its maker)No

Change Budget row

ColumnWhat changesDetailSource, read
CodeNo DataFrame or SQL changesGluten needs no DataFrame or SQL changes, only configuration.Apache Gluten getting started, 10 Nov 2026
ClusterPlugin, off-heap memory, shuffle managerThe settings are spark.plugins=org.apache.gluten.GlutenPlugin, off-heap memory enabled with a size, ColumnarShuffleManager and the bundle jar loaded ahead of Spark's jars.Apache Gluten getting started, 10 Nov 2026
CloudNone found on 2026-11-10No cloud requirement is stated. Official build and test platforms are Ubuntu 20.04/22.04 or CentOS 8, and the docs have pages for S3, GCS and ABFS storage.Apache Gluten downloads, 10 Nov 2026
VendorApache-2.0, no contract statedGluten's project license is Apache-2.0; no vendor or platform contract is described.Apache Gluten Powered By page, 10 Nov 2026

Searched for and none found on 10 Nov 2026

  • Any published cost per job or cost-for-the-same-work result (the software is free under Apache-2.0) (searched: gluten.apache.org, raw.githubusercontent.com/apache/gluten/main/docs/index.md, raw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-st)
  • Documented support or tested configuration for Amazon EMR, EKS or Databricks (searched: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta, gluten.apache.org/poweredby, raw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-st)
  • An independent benchmark of Gluten on AWS hardware (searched: gluten.apache.org, gluten.apache.org/poweredby, gluten.apache.org/community)

Apache Gluten with Velox side by side with each other option

Sources

  1. Apache Gluten with Velox: gluten.apache.org, read 10 Nov 2026
  2. Apache Gluten with Velox: raw.githubusercontent.com/apache/gluten/main/docs/get-started/getting-st, read 10 Nov 2026
  3. Apache Gluten with Velox: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/velox-b, read 10 Nov 2026
  4. Apache Gluten with Velox: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta, read 10 Nov 2026
  5. Apache Gluten with Velox: raw.githubusercontent.com/apache/gluten/main/docs/backends/velox/get-sta, read 10 Nov 2026
  6. Apache Gluten with Velox: gluten.apache.org/downloads, read 10 Nov 2026
  7. Apache Gluten with Velox: raw.githubusercontent.com/apache/gluten/main/tools/gluten-it/README.md, read 10 Nov 2026
  8. Apache Gluten with Velox: gluten.apache.org/poweredby, read 10 Nov 2026
  9. Apache Gluten with Velox: api.github.com/repos/apache/incubator-gluten/releases?per_page=3, read 10 Nov 2026

Last reviewed 10 Nov 2026.