Quickstart: Extract and transform your data into objects and events
The object-centric data model is system-agnostic. You can connect to any source system and use your data to create objects and events. Object-centric process mining overview explains more about how object-centric process mining works.
You can set up the whole of the object-centric data model, or choose one or more of the core processes - Accounts Payable, Accounts Receivable, Order Management, Procurement, and Inventory Management. We’ll enable the object types, event types, relationships, and perspectives for your selected processes. You can also start with a blank sheet if you want to build your own custom processes by creating object types and event types.
Celonis supplies prebuilt extractions and transformations for SAP ECC and Oracle EBS systems.
Extractions identify and ingest relevant business data from the tables in your source system.
Transformations create instances of objects, changes, events, and relationships based on the ingested data.
We’ll install the right transformations from the Celonis catalog when you set up a process and select a data connection for an SAP or Oracle system. For the extractions, we’ll soon have these available for automatic install, but for now ask your Celonis point of contact to help you set up the extraction package.
For other source systems, you can connect to your source system using the supplied extractor for your system if there is one, or create your own extractor using the Extractor Builder. Set up extraction tasks to bring in the data tables from your source system. (Or you can upload a sample of data from a file on your computer.) Then create your own SQL transformations for the data using our editor.
The objects and events created from your source system data, and the extractions and transformations, are all held in the OCPM Data Pool. We create this data pool when you access object-centric process mining for the first time. Having a single object-centric data model in the OCPM Data Pool ensures there is a single source of truth for your objects and events. You can opt to have a separate object-centric data model in each data pool in your Celonis team, if you require data segregation because of operational or compliance requirements. Talk to your Celonis point of contact if you think multiple object-centric data models might be right for you - Multiple object-centric data models has more on how that works.
Quickstart for SAP ECC and Oracle EBS
Follow these steps to use our prebuilt tasks to extract and transform data from an SAP ECC system or an Oracle EBS system. If you have any problems, check the solutions in Troubleshooting data extraction and pre-processing.
Stage 1: Create or import a Data Connection in the OCPM Data Pool
In the Celonis navigation bar, select Data > Objects and Events. When you do this for the first time, we create the OCPM Data Pool in your team. The OCPM Data Pool is where Celonis keeps the data, extractions, transformations, and perspectives for object-centric process mining. You can’t change its name.
On the dashboard (which you get to by selecting Dashboard in the top navigation bar), click the OCPM Data Pool button.
Tip
You can also get to the OCPM Data Pool from the Celonis navigation bar - select Data > Data Integration, and pick the OCPM Data Pool from the list of data pools.
If you have an existing Data Connection for this source system in another data pool, you can import it to the OCPM Data Pool and reuse it. Follow the instructions at Sharing data between data pools to export the Data Connection.
To set up a new Data Connection, in the Data Connections area, select the Add Data Connection button. If you’ve set up a Data Connection for Celonis before, you'll be familiar with the next few steps in this procedure.
Select Connect to Data Source to pick an appropriate data connection for your source system. In the On-Premise section, select the connector for SAP ECC systems or Oracle EBS systems.
Follow the steps in the Data Connection wizard to connect to your source system. If this is the first time you’ve connected to this source system, you need to configure an uplink on the system. If you need any help, see the system-specific advice for SAP connections or Oracle.
Click Test Connection to verify that your new Data Connection works.
After you’ve seen the confirmation message, click Save. The new Data Connection is created in the OCPM Data Pool.
Stage 2: Enable the processes you want
In the Celonis navigation bar, select Data > Objects and Events.
On the dashboard (which you get to by selecting Dashboard in the top navigation bar), click the Catalog button.
Or click Add from catalog in the list of processes.
Click the name of any of the Celonis processes, and use the Enable process slider to enable it. We’ll enable the Celonis object types, event types, relationships, and perspective for that process.
To add the Celonis transformations, select your data connection from the dropdown and click Add. If we haven’t identified the data connection, you’ll need to specify what transformations you want us to use for it. We’ll enable the SAP ECC or Oracle EBS transformations for each of the Celonis object types and event types in your selected process. You can add more than one data connection if you have multiple source systems.
Click the Done button to exit process setup. You can go back any time and enable further processes. You can also disable processes here.
Verify that the object types, event types, perspectives, and transformations for the process you installed are listed in the Objects and Events environment.
Tip
You can disable the transformation for any Celonis object type or event type if you don’t need it, or don’t have data for it, or are going to build it with data from a set of transformations for another source system. Use the toggle next to the transformation's name to disable it. You can re-enable the transformation at any time if you do need it in the future.
Select Publish > Publish to development in the top navigation bar. We’ll publish everything that’s now enabled to the OCPM Data Pool.
When you’ve set up the processes you want, ask your Celonis point of contact to help you set up the extraction package for them. We’ll have the extractions available for automatic install soon.
Stage 3: Run the extractions and transformations
When you’re ready to extract and transform the data from your source system, click the OCPM Data Pool button on the dashboard.
In the OCPM Data Pool, click the Data Jobs area.
Tip
If your extraction package was set up in a different data pool, click Data Pools in the breadcrumbs at the top left to go to the list of data pools, find that data pool, and follow the extraction instructions there.
Select the data job ocpm-extraction-job for the scope of the Data Connection for your source system. (Your extraction package’s data job might have a different name.) If you’re extracting data from more than one source system, you’ll need to repeat this for each source system.
In the list of extractions, select the name of any extraction if you want to review its details or make any configuration changes. Then select the Execute Data Job button to extract your data.
To run the extractions, click Full Load, then Execute Selection.
In the Data Jobs area, the extraction data job displays the status and execution time so far, and there’s a button if you want to stop the job running. Check the logs if there are any errors.
When all the extractions are complete, go to the Data Jobs area of the OCPM Data Pool, and select the data job test:ocpm-data-job, which is in the global scope. This data job runs the transformations to create test objects and events in your development environment.
The predefined transformations are in the data job ocpm-data-job. This data job is in the global scope for the data pool, and it’s where Celonis also stores custom transformations.
The development data job test:ocpm-data-job is the development version of ocpm-data-job. Use this data job to create test objects and events in your development environment, and use ocpm-data-job to create objects and events in your production environment.
Select the Execute Data Job button to transform your extracted data into objects and events.
Next steps
Customize and extend the object-centric data model for your business’s specific needs. Modeling objects and events tells you how to do this.
When you’ve set everything up, publish a read-only copy to the production environment by selecting Publish > Publish to development and production. Then run the data job ocpm-data-job to build your objects and events in production.
Set up a schedule to run your data pipeline regularly. See the section at the end of this page for more on scheduling.
Analyze your business processes using the objects and events you’ve built. Using objects and events for process mining tells you how to do this.
Quickstart for other source systems
Follow these steps to extract and transform data from a source system that’s not SAP ECC or Oracle EBS. If you have any problems, check the solutions in Troubleshooting data extraction and pre-processing.
Stage 1: Create or import a Data Connection in the OCPM Data Pool and extract your data
In the Celonis navigation bar, select Data > Objects and Events. When you do this for the first time, we create the OCPM Data Pool in your team. The OCPM Data Pool is where Celonis keeps the data, extractions, transformations, and perspectives for object-centric process mining. You can’t change its name.
On the dashboard (which you get to by selecting Dashboard in the top navigation bar), click the OCPM Data Pool button.
Tip
You can also get to the OCPM Data Pool from the Celonis navigation bar - select Data > Data Integration, and pick the OCPM Data Pool from the list of data pools.
If you have an existing Data Connection for this source system in another data pool, you can import it to the OCPM Data Pool and reuse it.
To set up a new Data Connection, in the Data Connections area of the OCPM Data Pool, select the Add Data Connection button.
If you’re uploading a file with sample data, select Upload files, and follow the instructions on the File Uploads panel. For a guide to that process, see Uploading data files.
If there’s a prebuilt extractor to connect to your source system, select Connect to Data Source, pick the extractor, then follow the steps in the Data Connection wizard. The instructions to set up each Celonis-supplied extractor are at Connecting data sources.
If there’s not an extractor for your source system yet, create a custom extractor using the Extractor Builder.
Click Test Connection to verify that your Data Connection works, then after you’ve seen the confirmation message, click Save.
To create a data job for your extraction tasks, in the Data Jobs area of the OCPM Data Pool, select the Add Data Job button. Name the data job and specify your Data Connection.
Select your new data job, and select the Add Tables button to create an extraction task. Add the tables that you need from your source system via your Data Connection, and configure any options and filters that you need for your data.
Run your data job to extract your source system data into the OCPM Data Pool.
Stage 2: Enable the processes you want
In the Celonis navigation bar, select Data > Objects and Events.
On the dashboard (which you get to by selecting Dashboard in the top navigation bar), click the Catalog button.
Or click Add from catalog in the list of processes.
Click the name of any of the Celonis processes, and use the Enable process slider to enable it. We’ll enable the Celonis object types, event types, relationships, and perspective for that process. You’ll create custom transformations in the next stage of setup.
Click the Done button to exit process setup. You can go back any time and enable further processes. You can also disable processes here.
Select Publish > Publish to development in the top navigation bar. We’ll publish everything that’s now enabled to the OCPM Data Pool.
Stage 3: Create custom transformations and transform your data
Follow the instructions in Creating custom transformations to create your own SQL transformations to map your extracted data to the object-centric data model. You’ll need to do this for the Celonis object types and event types in your selected processes, as well as for any custom types that you create.
When you’re ready to transform your data, select Publish > Publish to development to make sure you’ve picked up your latest changes to the object-centric data model and the transformations. We’ll publish your object types, event types, relationships, perspectives, and transformations to the OCPM Data Pool.
Go to the OCPM Data Pool by clicking the OCPM Data Pool button on the dashboard.
In the Data Jobs area, select the data job test:ocpm-data-job, which is in the global scope. This data job runs the transformations to create test objects and events in your development environment.
Select the Execute Data Job button to transform your extracted data into objects and events.
Next steps
Customize and extend the object-centric data model for your business’s specific needs. Modeling objects and events tells you how to do this.
When you’ve set everything up, publish a read-only copy to the production environment by selecting Publish > Publish to development and production. Then run the data job ocpm-data-job to build your objects and events in production.
Set up a schedule to run your data pipeline regularly. See the section at the end of this page for more on scheduling.
Analyze your business processes using the objects and events you’ve built. Using objects and events for process mining tells you how to do this.
Scheduling data jobs for object-centric process mining
When you've published your object-centric data model to the data pool, set up a schedule to run your data pipeline regularly. You'll need schedules for these data jobs:
ocpm-extraction-job, which extracts data from your source system. Your extraction package’s data job might have a different name. If you have multiple source systems, schedule the extractions for all of them.
ocpm-data-job, which runs the transformations to create objects and events in your production environment.
Any other data jobs that you’re using in your production environment for object-centric process mining.
The data job test:ocpm-data-job runs the transformations to create objects and events in your development environment. You might prefer to run this manually as you need to - the instructions are at Running transformations. You can set up a schedule if you prefer, for example to run it overnight.
If you're using only the OCPM Data Pool, set up the schedules in there. If you're using the feature that gives you object-centric data models in all your data pools, set them up in each data pool where you're using them. In the data pool, go to Schedules > Add Schedule and follow the instructions in Scheduling the execution of data jobs.
You can set up schedules to run the transformations for just one perspective at a time (or more than one if you want). Perspective scheduling can be done for Celonis catalog perspectives and custom perspectives.
Note
Perspective scheduling is currently in Limited Availability - if you'd like to try it out now, open a Support ticket to request it.
When you run the OCPM data job through a schedule where you've selected perspectives, we'll identify all the objects and events that are part of those perspectives. Then we’ll run only the transformations that are needed to create those objects and events from your source system data. If you have modeled several processes in the same data pool, you can use this method to limit the time taken and the potential for incomplete runs.
When queuing of data jobs is enabled, if an instance of a data job is running and another instance of the same data job is scheduled to start, the second instance will be queued until the first finishes. If queuing of data jobs isn't enabled, the second instance will be skipped.
Here's how to set up schedules for perspectives:
In the data pool where your object-centric data model is, add as many schedules as you want for ocpm-data-job or the development version test:ocpm-data-job.
In each schedule under Selected Data Jobs, go to the context menu (the three vertical dots) for your OCPM data job, and select the Configure Perspectives option.
Click Perspectives to display the list of perspectives that are enabled in your OCPM data job. Check the box for one or more perspectives that you want to load together and click Save.
To verify the results, you can run the schedule by clicking the Execute button at the top right, and confirming that you want to execute the OCPM data job.
Currently, while the feature is in Limited Availability, you'll also need to enable queuing of data jobs manually for every schedule where you've selected perspectives. To do this, go to the context menu (the three vertical dots) for each schedule and select Execution Settings, set the slider on, and click Save. When the feature enters General Availability, queuing of data jobs will become the default behavior for all schedules in the OCPM Data Pool.
Tip
Queuing of data jobs lets Data Integration schedule multiple instances of the same data job. It’s available in every data pool, not just the OCPM Data Pool. So you can activate it for other data jobs if you want to - including case-centric data jobs. In every data pool that doesn't have object-centric data models active, queuing of data jobs will be off by default. In every data pool that does, it will be on by default.