Data Engineering: ETL/ELT vs ETLT vs ETLT++

Material based on a scientific article about the hybrid ETLT pattern and its evolution toward ETLT++.

Igor Gorin · June 2026

Based on an article by Chiara Rucco, Motaz Saad, Antonella Longo. Original article

Introduction

In data engineering, data loading design patterns refer to standardized methods that include ETL/ELT processes, data pipelines, and data flow management.

At the same time, the modern data processing stack faces several systemic problems:

• A fragmented tool ecosystem and the complexity of interactions between tools.

• Complexity in organization and management.

• Persistent data quality issues that consume up to 80% of engineers’ time.

• Missing metadata, data lineage, data contracts, and similar governance components.

ETL and ELT

As shown in Figure 1, ETL/ELT methodologies transform data either before loading or after loading. The traditional ETL pattern applies business logic during transformation before the load step. ELT reverses this sequence by loading raw data directly into storage and performing transformations inside the warehouse. The ELT approach significantly reduces the load on the source system and the integration tool, allows transformation logic to be improved iteratively, and supports more flexible schema evolution.

Despite these advantages, ELT also has drawbacks:

• the absence of preliminary data quality checks, which can lead to the propagation of incorrect data,

• when ELT processes load data without preliminary filtering, they may load large volumes of unnecessary data and overload the transfer channel.

The Middle Ground

Theory is one thing, but in practice engineers often mix both approaches within a single flow depending on the situation and their own experience. The hybrid ETLT pattern is used constantly, yet there are no standardized definitions or established best practices for ETLT.

Classic ETL and ELT processes are mature reference processes. At the same time, the ETLT pattern remains underdeveloped. Going further, simply shifting ELT methods toward ETLT is not enough. We need patterns that perform data quality checks, provide auditability and governance, improve developer productivity, and systematize long-term maintenance. This leads to the next generation of patterns with the “++” suffix: ETLT++.

Moreover, the dynamic nature of data requirements, combined with the rapid emergence of new technologies, requires continuous evolution and adaptation of existing patterns.

1. ETLT Design Patterns

In many scenarios, the main difficulty lies in the quality of incoming data. Early-stage cleansing and validation are critical.

The ETLT pattern ensures data quality at the initial transformation stage, denoted as T1 in Figure 2. At this stage, cleansing, validation, and normalization are applied to ensure consistency. Only after this control step do the data move to the storage loading stage. Then, at the second transformation stage, T2, business transformation rules, enrichment, and schema formation for data marts are applied.

By moving data quality issues into T1, ETLT ensures that subsequent business transformations do not fail or become contaminated by low-quality input data.

This design also enables deterministic replay of T2 without re-extracting and re-cleansing the source data.

ETLT is especially useful when data quality cannot be taken for granted. This applies to systems that combine data from multiple sources or where source reliability is low. The pattern ensures that erroneous records are detected before they enter the central storage layer. This makes ETLT a valuable pattern for systems with strict data requirements, such as financial reporting or healthcare.

However, despite its strengths, the pattern does not imply mandatory enforcement of data contracts, deterministic replay, lineage collection, monitoring, or data quality metrics.

2. ETLT++

While ETLT separates data quality validation from business logic, ETLT++ adds advanced control metrics and functions. Its novelty lies in the shift from ad hoc hybrid workflows to a formally structured design pattern that ensures reproducibility, monitoring, and continuous quality assurance.

ETLT++ is defined as a sequence of connected stages:

P = ⟨E, C, T1 , L, T2 , O⟩

where:

• E (Extract from Sources): Raw data

• C (Data Contract Loading): A data contract. A rule object in JSON or another format. The contract specifies rules such as required fields, value ranges, formats, and rule severity (hard vs. soft).

• T1 (Validation and Cleaning): Applying the contract:

– If a hard rule is violated, the record is placed in quarantine; if a soft rule is violated, a warning is logged.

– Batch-level validation stops the entire load if any hard violations occur.

• L (Load into Versioned Storage): Storing validated records in the raw zone, which preserves every data load and timestamp.

• T2 (Business Logic and Transformation): Operations that transform raw data into structured, analysis-ready datasets, such as aggregations, enrichments, or historical change tracking using SQL transformation templates.

• O (Outputs): Publishing prepared datasets for downstream consumers.

2.1 Data Contracts (C)

In practice, one of the most critical problems in data engineering is that raw data coming from many heterogeneous sources is often incomplete or inconsistent. Without a blocking mechanism, such data enters storage and corrupts subsequent data layers. A single source-side error, such as a negative invoice amount, can propagate unchecked into financial dashboards and distort final figures.

Unlike ETLT, where validation mechanisms may be implicit or case-specific, ETLT++ defines checks specified in data contracts as mandatory and explicit safeguards. A data contract is a static specification of rules that every dataset must satisfy before entering the pipeline.

Data contracts are not new; they are a well-established data governance mechanism. However, in existing systems they are often optional or implemented inconsistently. In ETLT++, by contrast, data contracts are strictly mandatory.

Rules can be classified as hard or soft. Hard rules are strict constraints: if they are violated, the data must not enter the pipeline. Soft rules are advisory: violations generate warnings but do not block processing. This distinction provides flexibility.

2.2 Validation: Enforcing Contracts (T1)

The mandatory stage is validation, where the data contract is enforced.

Validation stages:

1. Contract extraction: loading the JSON contract from the metadata registry.

2. Record level. For each record r in the incoming batch:

   For each hard rule i and each record r, a violation indicator is calculated

   v_{i, r} =

   1 if record r violates rule i;

   0 if record r does not violate rule i.

   • If any (i, r) = 1, mark record r as quarantined. Continue validation.

   • If a soft rule is violated, log a warning but allow processing to continue.

3. Batch level. Calculate the total number of hard violations:

   If V > 0, stop loading the entire batch and mark it as failed; otherwise, proceed to loading.

   Example: suppose we receive a batch of five customer records:

Only record 1002 violates a hard rule and is placed in quarantine. Since V = 1, batch loading stops until the issue is resolved.

The effect of combining a contract with validation is that pipelines become predictable: invalid records are stopped at the boundary, warnings are logged for later review, and only trustworthy data enters storage.

In other words, in ETLT++ data quality is not an optional feature, but a mandatory property of a reliable modern data platform.

2.3 Loading and Versioning (L)

In many data pipelines, the loading phase is treated as a simple black-box operation in which data is saved to a database or lake. This oversimplification creates two recurring problems. First, many data stores in use do not support versioning natively, which means that once data is overwritten, previous states are lost forever. This makes it impossible to reconstruct what a dataset looked like at a specific point in time. Second, even when modern table formats such as Delta Lake, Apache Iceberg, or Hudi are available, versioning features are often poorly configured or ignored. In both cases, the consequence is the same: teams cannot perform time travel or conduct audits.

As a result, it becomes impossible to answer questions such as: “What did the data look like last week when the report was created?” or “Which records have changed since the last check?”

Therefore, ETLT++ treats versioned append-only loading as mandatory. Data is stored immutably: after records are inserted, they are never deleted or modified; they are only appended. This eliminates the fragility of ad hoc approaches and ensures that analysts, auditors, and engineers can always travel back in time across a dataset for replay and validation.

These properties provide reliability, transparency, and auditability for ETLT++ pipelines, raising ETL/ELT to a fundamentally higher level of quality.

3. Monitoring and Data Quality Assurance

Even with strict contracts, validation, and standardized transformations, pipelines can still produce errors, delays, or inconsistencies due to source-side failures, missing data, or human factors.

ETLT++ proposes a design pattern for monitoring and data quality assurance. The first step is to define Service Level Indicators (SLIs) that reflect the key aspects of quality: freshness, completeness, accuracy, and contract adherence. ETLT++ treats these SLIs as mandatory architectural components.

Freshness measures how up to date the data is. For example, if a system expects daily sales data but the latest batch was received three days ago, the freshness SLI will signal a problem.

Freshness = Current Time − Timestamp of Latest Batch

Completeness evaluates whether all expected records or fields have been received.

Completeness = Number of Records Received / Number of Records Expected

A low completeness score indicates missing or partial data.

Accuracy measures how well the data complies with the validation rules defined in the data contract. For example, if ages, prices, or dates fall outside expected ranges, accuracy decreases.

Accuracy = 1 − Invalid_records / Total Number of Records

Maintaining high accuracy ensures that the data used for reporting and decision-making is reliable enough.

Contract Adherence checks whether each incoming batch complies with the agreed data contract, including both hard rules that must be satisfied and soft rules that may generate warnings. Contract adherence can be monitored as the percentage of batches that fully comply with the contract:

Contract Adherence = Number of Compliant Batches / Total Batches

Tracking contract adherence provides visibility into whether upstream systems deliver data in the expected format and structure.

Once SLIs are defined, the pipeline is equipped with tools for automatically collecting metadata and statistics for each batch, including timestamps, record counts, and validation errors. Quality metrics are calculated for each dataset and compared against predefined SLO thresholds. If any metric falls below an acceptable level, automated alerts notify engineers or trigger corrective actions such as reloading or recalculation. Historical SLI logs are stored for audit and trend analysis. Cyclical analysis of SLI logs continuously improves data quality over time.

Examples of actions when indicators exceed thresholds:

• Freshness: check that the latest batch arrived within 24 hours.

• Completeness: confirm that all expected values and fields are present.

• Accuracy: ensure that transaction amounts are non-negative and within expected ranges.

• Contract Adherence: check that the schema matches the agreed definition and that required fields are present.

Embedding data quality directly into the pipeline as a reusable component based on ETLT++ patterns turns quality assurance from a one-time manual task into an active, automated, and scalable practice. This approach protects decision-making, maintains trust in analytics, and ensures reliable, reproducible processing of enterprise data.

Conclusion

ETLT++ turns ETLT into a reliable pattern by adding:

• Data Contracts

• Versioned raw storage: immutability, traceability, and reproducibility by design.

• Rewindable business logic: deterministic transformations that can be reapplied.

• Continuous monitoring: SLIs and SLOs integrated into the pipeline fabric.

These properties provide reliability, transparency, and auditability for ETLT++ pipelines, raising ETL/ELT to a fundamentally higher level of quality.