Data pool architecture with the Replication Cockpit
When using the Replication Cockpit to establish real-time connectivity to your SAP systems, you have several options on how to configure your data pipelines and the data pool you're using.
Real-time extractions are based on change logs in the SAP system, meaning that every successful extraction of data is processed based on the information in the change log. This means that each table can only be extracted once via the Replication Cockpit. If you attempt to extract one table via multiple data pools, the replications may negatively impact each other and data may be missing. As such, we recommend having a single data pool that extracts the data via the Replication Cockpit.
To understand how to transfer data between data pools, see: Sharing data between data pools.
Use case 1: Using real-time extractions but no real-time transformations
Single extraction data pool: In this scenario, we have one single data pools that contains all data connections as well as all data extractions from the source systems. This means the extracted tables for all processes are being combined in this pool which serves as a central storage of all raw data.
Separate transformation data pools: For every single process or use case, there is a separate data pool that contains all data transformations and the data models built on top.
Export/Import: This option leverages the data transfer (Export/Import) functionality on a data pool level. It allows you to export data from one pool to another in the form of database views that are being created.
Impact:
Single extraction data pool allows full support of real-time extractions in the Replication Cockpit.
Use cases are still divided in pools and permissions can be managed on a use case level
Real-time transformations are not possible as this requires extractions and transformations to reside in the same data ool.
Use case 2: Using real-time extractions and real-time transformations
Single Global Data Pool: In this scenario, we have one single Data Pool that contains everything, meaning all Data Connections, all Data Extractions, and all Data Transformations for all processes and use cases. There is no separation of use cases on Data Pool level anymore. Such distinction can be only achieved through different Data Jobs and Data Models.
Support for Real-Time Transformations: This set-up is the only one that comes with the full support of Real-Time Transformations. The transformations in Replication Cockpit are directly coupled to the extractions, which requires both to be executed in the same Data Pool.
Impact:
One single data pool covering all parts of the data pipeline.
Full real-time connectivity to SAP incl. real-time extractions and real-time transformations.
No clear separation of use cases.