ETL (Extract-Transform-Load)

ETL vs ELT

• Extract → Get data from various sources (APIs, databases, sensors, etc.)

• Transform → Clean, enrich, or aggregate the data

• Load → Store the processed data into a database or data warehouse

ETL = Kafka + Spark + Cassandra

[Producers]

     ↓

 Kafka topics  ← (Extract)

     ↓

 Spark Streaming job  ← (Transform)

     ↓

 Cassandra tables  ← (Load)

     ↓

[Dashboards, APIs, Analytics]

• OLTP = Online Transaction Processing

• OLAP = Online Analytical Processing