I. INTRODUCTION
Azure Data Factory is the cloud-based ETL and data integration service.
- Using ADF, we can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores.
- We can build complex ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop, Azure Databricks, and Azure SQL Database.
- Additionally, we can publish our transformed data to data stores such as Azure Synapse Analytics for business intelligence (BI) applications to consume.
- Ultimately, through ADF, raw data can be organized into meaningful data stores and data lakes for better business decisions.
II. HOW DOES IT WORK
Data Factory contains a series of interconnected systems that provide a complete end-to-end platform for data engineers.
1. Connect and collect: Enterprises have data of various types that are located in disparate sources on-premises, in the cloud, structured, unstructured, and semi-structured, all arriving at different intervals and speeds.
