Extractions Editor and AI Assistant overview
Extraction tasks allow you to select the data tables to be extracted from your source system (and imported into the Celonis Platform). You can now create, validate, and preview your extraction tasks using the Extractions Editor and AI Assistant allowing you to dynamically write SELECT SQL queries and use your source system's SQL dialect.
For guidance on using the Extractions Editor and AI Assistant, see: Creating extraction tasks using the Extractions Editor and AI Assistant.
And for further details about creating extraction tasks using the existing visual editor, see: Creating extraction tasks using the visual editor
And for a video demo of this feature:
What are the Extractions Editor and AI Assistant?
The Extractions Editor and AI Assistant offer full control to the users who are configuring extractions. You can now write (or generate) an extraction query as you would on your source system to fetch data. This means more power and flexibility for you, allowing you to use your source system specific SQL functions and JOINS. The AI Assistant not only generates the query but also helps in validating a written query by detecting syntax errors.
What challenges are the Extractions Editor and AI Assistant solving for you?
Source systems have multiple tables housing various objects and events from different processes. And when connecting to your JDBC databases, our current wizard based approach helps you configure your extractions. While this is relatively easy to use, there are also trade offs for you:
You could only extract one table at a time, resulting in a larger number of transformations before creating the required data model.
You had limited data filtering capability because of negligible support for specific source system SQL dialect.
You needed to traverse a page long GUI with multiple inputs to configure an extraction.
What are the benefits of using the Extractions Editor and AI Assistant?
Flexibility and control: Using the Extractions Editor, you configure extractions by simply writing a SELECT SQL Query leveraging the SQL dialect of your source system. You join multiple tables existing on your source system already on the extraction phase thereby reducing the need for multiple transformations.
Time savings: There's now a step reduction in extracted data volume and the time to data model creation.
AI Assistant: SQL queries can also be generated and validated using our AI Assistant . Very useful for those who'd like to transfer the task of writing the actual SQL query (remember those cryptic column names on your tables?) all by yourself. The AI Assistant has the context of your tables and happily generates the query for you.
Is the Extractions Editor replacing the current visual (wizard based) configuration?
Not in the short term. The wizard based configuration will stay as it is for now. We will notify in due course of time when we want to deprecate the visual mode.
What is the difference between an extraction created using visual mode vs an extraction created using the Extractions Editor?
When you create an extraction using visual mode, you can add several source system tables as a part of that task. Next, you have to configure each of those tables separately using the visual wizard of their own. During the extraction execution, all the tables included in the configuration are extracted to vertica, based on the way you entered inputs during the visual mode.
On the other hand, the extraction task created using the Extractions Editor is “one SELECT Query”. This may refer to a SELECT statement pertaining to one table or you could use a JOIN and combine data from multiple tables. In essence, one extraction task results in one target table in Vertica.
Can I switch between the two modes?
No, both extraction modes work independently of each other as far as configuration is concerned. And extraction created using one mode cannot be converted to another.
However, given the properties of your target table you can replicate a visual mode extraction using the Extractions Editor. For example - you have four tables that you are extracting from your source system. Then you are eventually joining those tables during transformations into one table in Vertica. You can use the Extractions Editor to create one extraction referring to the eventual target table.
Are there any prerequisites before using the Extractions Editor and AI Assistant?
If you're currently connecting to your source system using the JDBC extractor, you must update the extractor to version 3.0.0 in order to use the Extractions Editor and AI Assistant.
For more information about installing and updating the JDBC extractor to version 3.0.0, see: Updating the on-premise JDBC extractor.
How can I configure delta extractions using the Extractions Editor?
You can create a dynamic parameter and use that in the SQL query. As a result the SELECT statement will control what you want to extract from your source tables.
What is a target table?
This refers to the table in vertica that stores your extraction data. In contrast to visual mode, where you could configure the target table as a part of the wizard, the Extractions Editor allows you to to configure the target table once you have finalized your query. With the Extractions Editor, you have to provide a name for the target table.
Using the Extractions Editor, how do I type-cast the columns and anonymise the data?
With the Extractions Editor, the recommendation is to use SQL query to perform these functions.
Is the AI Assistant provided with the Extractions Editor or I can use them separately?
You can choose to enable the Extractions Editor without the AI Assistant. However, it doesn’t work the other way around.
Is there an additional charge for using the Extractions Editor and AI Assistant?
There is no additional charge for using the Extractions Editor and AI Assistant.
Do I need to sign a consent form to use the Extractions Editor and AI Assistant?
Yes. During the Limited Availability phase, you do need to sign a consent form to use the Extractions Editor and AI Assistant. To do this, reach out to your Celonis account manager.