Skip to main content

Delta Sharing connections

This feature is currently available as a Private Preview only

During a Private Preview, only customers who have agreed to our Private Preview usage agreements can access this feature. Additionally, the features documented here are subject to change and / or cancellation, so they may not be available to all users in future.

If you would like to use this feature, create a Support ticket at Celonis Support.

For more information about our Private Preview releases, including the level of Support offered with them, see: Feature release types.

You can establish zero-copy connectivity between your data platform and the Celonis Platform, allowing you to share data to Celonis with near-zero latency. This means that any updates in your data lake are automatically available in your Celonis data pool.

The benefits of this connection include:

  • Zero-copy data access: Query your raw data directly at the source without duplication. We leverage Open Delta Sharing as a standardized zero-copy technology.

  • Simplified setup: No complex extraction pipelines. Copies of raw data are eliminated. Connect once and start consuming data immediately.

  • Native integration in your IT landscape: Raw data remains in its original location, preserving your organization's data governance.

To see a demonstration of establishing a connection and sharing data using Delta Sharing, view the following video:

 

Prerequisites

This section details important prerequisites or prerequisite knowledge for using Delta Sharing:

  • Your Data Pool must be using the ETL Engine. For more details, see ETL Engine.

  • You must create a data share in your source system using Open Delta Sharing and bearer token. This data share must define the tables and views you want to share with the Celonis Platform.

    When creating this data share, your IT admin should receive a credential file or a download link. It will contain information required to create the data connection in Celonis.

    Note

    You must have this information before starting Configuring Delta Sharing connection.

    For more information, see your source system documentation:

  • The Delta Sharing endpoint of the source platform needs to be accessible from Celonis. For more information, see Allowlisting Celonis domain names, IP addresses, and third-party domains.

  • The underlying object storage files must be accessible from Celonis.

Note

Refer to Known issues and limitations and FAQs for additional information.

Supported platforms

Delta Sharing is supported for the following sources:

  • Databricks

  • Jira

  • Platforms that supports Open Delta Sharing

Supported authentication types

Delta Sharing currently supports bearer token authentication.

Configuring Delta Sharing connection

This section describes the basic setup of a Delta Sharing connection. To configure the connection:

  1. In the Celonis Platform left navigation, select Data > Data Integration.

  2. On the Data Pools screen, select the data pool where you want to integrate the extractor. This opens the Data Integration window.

    Note

    Your Data Pool must be using the ETL Engine. For more details, see ETL Engine.

  3. In the Input section, select the Connect to Data Source tile. This opens the Add Data Connection window.

    Note

    If this is not the data pool's first connection, the Data Connections window opens below. Select + Add Data Connection to add a new connection.

  4. In the Add Data Connection window, select Connect to Data Source. This opens the in-tool connection catalog.

  5. In the connection catalog, select the Delta Sharing tile. This opens the configuration window.

  6. In the configuration window, provide the requested data for each field.

    Note

    If you upload a Credential file, the Server URL and Bearer Token are automatically populated. Optionally, you can manually enter the required data in these fields.

  7. Select the Authenticate button to load the available shares.

  8. Select the desired share.

  9. Select Save.

You can now see the source tables available in the Schema Explorer. Raw data is retrieved from the source system at transformation runtime and the transformation is then executed in the Celonis Platform. Tables that are created through transformations (e.g. event logs) are persisted in Celonis.

Known issues and limitations

This section describes the known issues and limitations for using Delta Sharing:

  • When Databricks views are shared with the Celonis Platform, updates on the source data may not be reflected immediately due to caching on the Databricks side.

FAQs

This section describes basic answers to frequently asked questions:

  • What change does a zero-copy integration imply for my data pipeline in Celonis?

    • Extraction: Extractions are completely eliminated. You do not need to configure or schedule extractions anymore.

    • Transformation: For transformations, you can reference the source system’s raw data directly without ingesting it to Celonis before. Raw data is retrieved from the source system when the transformation runs, and the transformation is then executed on Celonis. Tables that are created through transformations (e.g. event logs) are persisted in Celonis.

    • Data Model Load: The Data Model Load remains unchanged.

  • Where are the transformations executed and computed?

    The computation of transformations is done on the Celonis Platform. With every query, that raw data is retrieved from the source system.

    Note

    Whether filters are propagated to the source system depends on the underlying platform.

  • Are there any additional costs on my data platform when using zero-copy with Celonis?

    No extra compute costs are generated on your platform, as Celonis executes all compute operations.

    Important

    Although no extra compute costs are generated by using Delta Sharing, depending on your cloud provider, the region of your data platform/object storage, and your Celonis team’s cluster, standard network egress (data transfer) rates may apply.