Side by side, read 10 Nov 2026
Flarion and Google Cloud Lightning Engine: the published facts in the same rows
Flarion profileGoogle Cloud Lightning Engine profile
The same rows for both
| Row | Flarion | Google Cloud Lightning Engine |
|---|---|---|
| Kind | Rust and DataFusion engine as one JAR | Premium Spark engine on Google Cloud |
| Does well | Flarion lists prices: $0.009 per CPU hour for self-hosted Spark, $0.012 for fully managed Spark such as EMR, $0.095 per DBU on Databricks and $0.75 per CPU per month on premises. Its benchmarks page publishes the machine, Spark 3.5.5 baseline, engine builds, memory budgets and per-query caps, and discloses that q72 is counted at vanilla Spark's time for Gluten and Comet. | Google's documentation says no Spark code changes are needed; clusters use image 2.3.3 or later, or 3.0.2 or later, and the flag --engine=lightning. Google publishes rates: $0.06 per DCU-hour standard and $0.089 premium in us-central1, with serverless billed per second. |
| Limited | Flarion is proprietary and not downloadable (the page says contact us), and no repository or scripts were found for its benchmarks. A published list of supported Spark versions: none found on 2026-11-10. Third-party benchmarks of Flarion: none found on 2026-11-10. | Lightning Engine is Google Cloud only, and the 3.6x launch figure comes with no scripts, cluster spec or raw timings. Per Google's documentation, Native Query Execution is not recommended for RDD-heavy jobs, UDFs, most Spark ML libraries or I/O-bound jobs, and NQE jobs on GPU accelerators fail. |
| Published speed results | The home page headline is 'Run faster, spend less, scale further, without changing a line of code', with tiles for 6 minute setup, 3x performance gains and 60% cost reduction. The tiles carry no workload or method on the page. The benchmarks page (September 2026) compares six engines on TPC-DS (103 queries) on one i4i.8xlarge reading Parquet from S3. Flarion is faster than vanilla Spark, Comet and Gluten, and slower than Databricks Photon in both scales. At 100 GB: Spark 1,135 s, Flarion 386 s, Photon 362 s. At 1 TB: Spark 135.6 min, Flarion 38.3 min, Photon 30.4 min. The page labels these as benchmark figures, not customer results, and not an audited TPC result. | Google's launch blog says that at a 10 TB dataset size Lightning Engine accelerates Spark by 3.6x on TPC-H-like workloads versus open source Spark on similar infrastructure. The queries are derived from TPC-H and not comparable to official TPC-H results. The pricing page says Lightning Engine delivers up to 4.9x faster performance than open source Spark and up to 2.5x the price-performance over a high-speed Spark offering Google does not name, with no workload, method or competitor named on the page. The docs claim Native Query Execution on Google Axion ARM (C4A) delivers 2.5x to 4x speedup over open source Spark on analytical workloads. No method is given on the page. |
| Published cost results and prices | Usage-based list prices: self-hosted Spark on AWS, Azure or GCP $0.009 per CPU hour; fully managed Spark (EMR, GCP managed Spark, Azure HDInsight) $0.012 per CPU hour; Databricks $0.095 per DBU; on-premises $0.75 per CPU per month. Billing is on accelerated Spark cores (driver plus executors). A free assessment simulates each job from logs and reports projected runtime and compute. | Serverless is billed per second in Data Compute Units (DCU, 1-minute minimum). Example us-central1 rates: standard $0.06 per DCU-hour, premium (Lightning Engine) $0.089 per DCU-hour. Clusters pay a management fee plus a Lightning Engine add-on plus Compute Engine VMs and disks. |
| Code changes | Flarion is one JAR inside the existing cluster, with no agents or separate services. UDFs and UDAFs run on Spark as before. Operations Flarion does not accelerate run on Spark automatically. The page lists a 'one config change' install and says you can turn it off with one config change to compare. | Docs say no Spark code changes. Requires image 2.3.3+ or 3.0.2+ (debian13/ubuntu24 for Lightning Engine) and enabling with --engine=lightning at cluster creation. Spark, PySpark, SparkSQL and SparkR jobs are accelerated; other job types run on the standard engine. |
| Tuning | The FAQ says existing Spark tuning carries over and is kept; Flarion changes how the engine executes. The garbage-collection post says native memory is managed off-heap with no GC pauses, but part of a job (driver, planning, unsupported operators, often the final write) still runs on the JVM heap. Telemetry marks each stage native or Spark with runtime and memory. | NQE is not recommended for RDD-heavy jobs, UDFs, most Spark ML libraries or I/O-bound jobs. NQE jobs on GPU accelerators fail. Unsupported inputs include Byte in ORC and Parquet and Struct/Array/Map in Parquet. A qualification tool exists to check jobs. |
| Spark versions, platforms and table formats | Platforms listed: Databricks (init scripts), Amazon EMR (bootstrap action), Google Cloud Dataproc, Azure HDInsight, Spark on Kubernetes (Helm chart or Spark operator) and on-premises. Frameworks: Scala Spark, PySpark, Trino, Ray. Table formats: Iceberg, Delta Lake, Hudi. File formats: Parquet, Avro, JSON. The benchmark ran Spark 3.5.5 and Databricks 16.4 LTS (Spark 3.5.2). A supported-version list is not published; a blog post mentions staying on Spark 3.4 and gaining newer-Spark benefits. | NQE reads Parquet, ORC, Apache Iceberg and Delta Lake. Hudi is covered by general Dataproc docs but not stated for NQE. Runs on x86 Intel and AMD, and ARM Google Axion C4A machines. |
| Where it runs | No special hardware: Flarion runs CPU-native on the fleet already in use (FAQ contrasts it with GPU accelerators). It is usable on AWS, Azure, GCP, on-premises and air-gapped setups. The company is SOC 2 Type II certified per the home page. | Google Cloud only. Rates on the pricing page are regional (us-central1 example given). |
| How open its benchmarks are | The benchmark page publishes the machine, Spark 3.5.5 baseline, engine builds (Comet 1.0.0, Gluten 1.7.0 Velox, Databricks DBR 16.4 LTS), memory budgets, shuffle partitions (64), AQE on, no statistics/CBO, per-query caps (600 s at 100 GB, 1,800 s at 1 TB) and lower of two runs at 100 GB and one run at 1 TB. It discloses that q72 is counted at vanilla Spark's time for Gluten and Comet. It says the open-source engines can be rerun from public releases, but Flarion itself is not downloadable ('contact us'), and no repo or scripts were found. | The launch blog footnotes the TPC-H-like derivation but publishes no scripts, cluster spec or raw timings. |
| Change Budget: code | No code change stated. Flarion says UDFs and UDAFs run on Spark as before and operations it does not accelerate run on Spark automatically. The home page headline says 'without changing a line of code'. | No Spark code change stated. Google's documentation says Lightning Engine needs no Spark code changes. Spark, PySpark, SparkSQL and SparkR jobs are accelerated; other job types run on the standard engine. |
| Change Budget: cluster | One JAR and one config change. Flarion is one JAR inside the existing cluster, installed with a one config change and switched off with one config change. Existing Spark tuning carries over. | Image 2.3.3+ or 3.0.2+, engine flag. Clusters need image 2.3.3 or later, or 3.0.2 or later (debian13 or ubuntu24), and the flag --engine=lightning at creation. NQE jobs on GPU accelerators fail. |
| Change Budget: cloud | No single cloud required. Flarion lists Databricks, Amazon EMR, Google Cloud Dataproc, Azure HDInsight, Spark on Kubernetes and on-premises, and says it supports air-gapped setups. | Google Cloud only. Google lists Lightning Engine on Managed Service for Apache Spark, with regional rates (us-central1 as the example). |
| Change Budget: vendor | Flarion license at usage-based prices. Flarion lists $0.009 per CPU hour (self-hosted), $0.012 (fully managed Spark), $0.095 per DBU (Databricks) and $0.75 per CPU per month (on premises), billed on accelerated Spark cores. | Premium tier or cluster add-on. Serverless premium is $0.089 per DCU-hour against $0.06 standard in us-central1. Clusters pay a management fee plus a Lightning Engine add-on plus Compute Engine VMs and disks. |
| Ledger entries | Flarion TPC-DS SF100, six engines on one machine (run by its maker); Flarion TPC-DS SF1000, six engines on one machine (run by its maker) | Lightning Engine against open source Spark, TPC-H-like 10 TB (run by its maker); Lightning Engine claim on Google's pricing page (run by its maker); Native Query Execution on Axion ARM, Google's documentation (run by its maker) |
How to read this page
The rows hold what each vendor or a named third party published, with no conclusion drawn. Benchmarks in the two columns were run on different hardware, data and baselines unless the Ledger entry says otherwise, so the numbers are not comparable across columns. The Ledger method explains how entries are recorded.
Sources
- Flarion: flarion.io, read 10 Nov 2026
- Flarion: flarion.io/benchmarks, read 10 Nov 2026
- Flarion: flarion.io/pricing, read 10 Nov 2026
- Flarion: flarion.io/faqs, read 10 Nov 2026
- Flarion: flarion.io/product, read 10 Nov 2026
- Google Cloud Lightning Engine (Managed Service for Apache Spark): cloud.google.com/blog/products/data-analytics/introducing-lightning-engi, read 10 Nov 2026
- Google Cloud Lightning Engine (Managed Service for Apache Spark): docs.cloud.google.com/managed-spark/pricing, read 10 Nov 2026
- Google Cloud Lightning Engine (Managed Service for Apache Spark): docs.cloud.google.com/managed-spark/docs/guides/lightning-engine, read 10 Nov 2026
Last reviewed 10 Nov 2026.