Data Management

Data Management within the platform is a control panel for loaded data packages that allows you to monitor and maintain data consistency and quality in tables.

When loading data between different systems, the data is divided into packages, with each package marked by its own unique number: Request ID or Batch ID. Within a single system, during loads driven by transformation flows, this unique number remains unchanged. This unique number allows you to manage loaded data point-by-point and track load statuses for operational data integrity control across the chain.

Deleting a specific Request ID / Batch ID

If a failure occurs during the loading process, data becomes corrupted, or an invalid package arrives from the source, there is no need to clear the entire table. You just need to find the problematic Request ID in the Data Management panel and delete only that specific package. Note that packages can only be deleted starting from the latest one. That is, you cannot delete a package from the middle, as this would break data consistency. However, you can delete multiple packages at once "from the top." To delete one or several packages, navigate to the Data management panel, load the table containing package metadata, select the packages in the table, and click the Delete button.

Deleting the entire table

In cases where the transformation logic has changed completely, a global failure has occurred, or a full recalculation of historical data is required, you can clear the target table entirely (Truncate) and restart the flow chain from the very beginning. To delete all data from a table, navigate to the Data management panel, load the table containing package metadata, do not select any packages in the table, and click the Delete button.