# Google Cloud Lightning Engine and Microsoft Fabric native execution engine: the published facts in the same rows

Side by side, read 10 Nov 2026

**The short answer.** Google Cloud Lightning Engine and Microsoft Fabric native execution engine are set out here in the same rows: published speed and cost results, code and cluster changes, tuning, stack fit and where each runs. The Ledger holds 3 entries for Google Cloud Lightning Engine and no entries for Microsoft Fabric native execution engine.

## The same rows for both

| Row | Google Cloud Lightning Engine | Microsoft Fabric native execution engine |
| --- | --- | --- |
| Kind | Premium Spark engine on Google Cloud | Velox-based engine inside Fabric Spark |
| Does well | Google's documentation says no Spark code changes are needed; clusters use image 2.3.3 or later, or 3.0.2 or later, and the flag --engine=lightning. Google publishes rates: $0.06 per DCU-hour standard and $0.089 premium in us-central1, with serverless billed per second. | Per Microsoft's documentation, no code modifications are required, and the engine is enabled with an Acceleration tab toggle or spark.native.enabled. The engine works with Parquet, Delta and CSV, including data reached through OneLake shortcuts. |
| Limited | Lightning Engine is Google Cloud only, and the 3.6x launch figure comes with no scripts, cluster spec or raw timings. Per Google's documentation, Native Query Execution is not recommended for RDD-heavy jobs, UDFs, most Spark ML libraries or I/O-bound jobs, and NQE jobs on GPU accelerators fail. | The engine runs only inside Microsoft Fabric on Azure and does not run on AWS EMR or EKS. Microsoft's page gives no number, workload or dataset for the speedup, and a benchmark method or scripts: none found on 2026-11-10. |
| Published speed results | Google's launch blog says that at a 10 TB dataset size Lightning Engine accelerates Spark by 3.6x on TPC-H-like workloads versus open source Spark on similar infrastructure. The queries are derived from TPC-H and not comparable to official TPC-H results. The pricing page says Lightning Engine delivers up to 4.9x faster performance than open source Spark and up to 2.5x the price-performance over a high-speed Spark offering Google does not name, with no workload, method or competitor named on the page. The docs claim Native Query Execution on Google Axion ARM (C4A) delivers 2.5x to 4x speedup over open source Spark on analytical workloads. No method is given on the page. | The page says the engine significantly raises query performance while minimizing operational costs, but gives no number, workload or dataset. 'Actual results vary by workload characteristics and configuration.' |
| Published cost results and prices | Serverless is billed per second in Data Compute Units (DCU, 1-minute minimum). Example us-central1 rates: standard $0.06 per DCU-hour, premium (Lightning Engine) $0.089 per DCU-hour. Clusters pay a management fee plus a Lightning Engine add-on plus Compute Engine VMs and disks. | Fabric is billed by capacity units (F2 to F4096 SKUs), pay-as-you-go or reservation. An opt-in Autoscale Billing for Spark runs Spark jobs on dedicated serverless resources at standard regional pay-as-you-go CU-hour rates, with a base Fabric capacity still required. SKU prices on the page load dynamically and were not readable. |
| Code changes | Docs say no Spark code changes. Requires image 2.3.3+ or 3.0.2+ (debian13/ubuntu24 for Lightning Engine) and enabling with --engine=lightning at cluster creation. Spark, PySpark, SparkSQL and SparkR jobs are accelerated; other job types run on the standard engine. | Docs say no code modifications are required and no vendor lock-in. Unsupported operators fall back to JVM Spark. Listed limitations include decimal-to-float casting, timezone, round(), map() duplicate-key and collect_list ordering differences on Runtime 1.3, said to be resolved in Runtime 2.0. |
| Tuning | NQE is not recommended for RDD-heavy jobs, UDFs, most Spark ML libraries or I/O-bound jobs. NQE jobs on GPU accelerators fail. Unsupported inputs include Byte in ORC and Parquet and Struct/Array/Map in Parquet. A qualification tool exists to check jobs. | Enable via the Acceleration tab toggle in an environment, or set spark.native.enabled to true at the start of a notebook or Spark job definition. Per-query disable is by setting the flag to false and re-enabling in later cells. Fabric Spark Advisor alerts show fallback. |
| Spark versions, platforms and table formats | NQE reads Parquet, ORC, Apache Iceberg and Delta Lake. Hudi is covered by general Dataproc docs but not stated for NQE. Runs on x86 Intel and AMD, and ARM Google Axion C4A machines. | Supports Runtime 1.3 (Spark 3.5, Delta Lake 3.2) and Runtime 2.0 (Spark 4.1, Delta Lake 4.1). Works with Parquet, Delta and CSV, including data reached via OneLake shortcuts. Not an AWS service. |
| Where it runs | Google Cloud only. Rates on the pricing page are regional (us-central1 example given). | Runs only inside Microsoft Fabric (Azure) on OneLake data. It does not run on AWS EMR or EKS. |
| How open its benchmarks are | The launch blog footnotes the TPC-H-like derivation but publishes no scripts, cluster spec or raw timings. | No benchmark method, data or scripts are published for the engine's speedup on the documentation page. |
| Change Budget: code | No Spark code change stated. Google's documentation says Lightning Engine needs no Spark code changes. Spark, PySpark, SparkSQL and SparkR jobs are accelerated; other job types run on the standard engine. | No code modifications stated. Microsoft's documentation says no code modifications are required. Unsupported operators fall back to JVM Spark. |
| Change Budget: cluster | Image 2.3.3+ or 3.0.2+, engine flag. Clusters need image 2.3.3 or later, or 3.0.2 or later (debian13 or ubuntu24), and the flag --engine=lightning at creation. NQE jobs on GPU accelerators fail. | Toggle in environment or Spark config. Enable it with the Acceleration tab toggle in an environment, or set spark.native.enabled to true in a notebook or Spark job definition. It needs Runtime 1.3 or 2.0. |
| Change Budget: cloud | Google Cloud only. Google lists Lightning Engine on Managed Service for Apache Spark, with regional rates (us-central1 as the example). | Microsoft Fabric on Azure only. The engine runs only inside Microsoft Fabric on OneLake data, and does not run on AWS EMR or EKS. |
| Change Budget: vendor | Premium tier or cluster add-on. Serverless premium is $0.089 per DCU-hour against $0.06 standard in us-central1. Clusters pay a management fee plus a Lightning Engine add-on plus Compute Engine VMs and disks. | Fabric capacity required. Fabric is billed by capacity units (F2 to F4096 SKUs), pay-as-you-go or reservation. Autoscale Billing for Spark still requires a base Fabric capacity. SKU prices were not readable on the page. |
| Ledger entries | [Lightning Engine against open source Spark, TPC-H-like 10 TB](/ledger/google-lightning-engine-tpch-like-10tb/) (run by its maker); [Lightning Engine claim on Google's pricing page](/ledger/google-lightning-engine-pricing-page-claim/) (run by its maker); [Native Query Execution on Axion ARM, Google's documentation](/ledger/google-native-query-execution-axion/) (run by its maker) | None found on 10 Nov 2026. |

Options in alphabetical order. Vendor statements are the vendor's own. Read 10 Nov 2026.

## How to read this page

The rows hold what each vendor or a named third party published, with no conclusion drawn. Benchmarks in the two columns were run on different hardware, data and baselines unless the Ledger entry says otherwise, so the numbers are not comparable across columns. The [Ledger method](/ledger/method/) explains how entries are recorded.

## Sources

- Google Cloud Lightning Engine (Managed Service for Apache Spark): cloud.google.com/blog/products/data-analytics/introducing-lightning-engi: https://cloud.google.com/blog/products/data-analytics/introducing-lightning-engine-for-apache-spark (read 10 Nov 2026)
- Google Cloud Lightning Engine (Managed Service for Apache Spark): docs.cloud.google.com/managed-spark/pricing: https://docs.cloud.google.com/managed-spark/pricing (read 10 Nov 2026)
- Google Cloud Lightning Engine (Managed Service for Apache Spark): docs.cloud.google.com/managed-spark/docs/guides/lightning-engine: https://docs.cloud.google.com/managed-spark/docs/guides/lightning-engine (read 10 Nov 2026)
- Microsoft Fabric Spark native execution engine: learn.microsoft.com/en-us/fabric/data-engineering/native-execution-engin: https://learn.microsoft.com/en-us/fabric/data-engineering/native-execution-engine-overview (read 10 Nov 2026)
- Microsoft Fabric Spark native execution engine: azure.microsoft.com/en-us/pricing/details/microsoft-fabric: https://azure.microsoft.com/en-us/pricing/details/microsoft-fabric/ (read 10 Nov 2026)

Last reviewed 10 Nov 2026. Canonical: https://sparkaccelerationcompare.com/compare/dataproc-lightning-engine-vs-fabric-native-execution-engine/
