From 2f20c529f4306c4dccedb13a4be461f06d62e572 Mon Sep 17 00:00:00 2001 From: Jonny Dixon <45681293+acrylJonny@users.noreply.github.com> Date: Mon, 2 Dec 2024 18:08:20 +0000 Subject: [PATCH] fix(docs): Add spark.datahub.stage_metadata_coalescing to recommended configuration for databricks (#11800) --- metadata-integration/java/acryl-spark-lineage/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/metadata-integration/java/acryl-spark-lineage/README.md b/metadata-integration/java/acryl-spark-lineage/README.md index 267e979b0fa073..97851e90e860ed 100644 --- a/metadata-integration/java/acryl-spark-lineage/README.md +++ b/metadata-integration/java/acryl-spark-lineage/README.md @@ -125,9 +125,10 @@ information like tokens. configurations under `Spark Config`. ```text - spark.extraListeners datahub.spark.DatahubSparkListener - spark.datahub.rest.server http://localhost:8080 - spark.datahub.databricks.cluster cluster-name + spark.extraListeners datahub.spark.DatahubSparkListener + spark.datahub.rest.server http://localhost:8080 + spark.datahub.stage_metadata_coalescing true + spark.datahub.databricks.cluster cluster-name ``` - Click the **Init Scripts** tab. Set cluster init script as `dbfs:/datahub/init.sh`.