In this article we will have two real time demo on how Azure Data Factory Data Flow implements Change Data Capture feature. Azure Data Factory and Azure Synapse Mapping Data Flow implements CDC in two ways : Let’s understand below details first: Let’s get connected: Native SQL Change Data Capture feature in Azure Data Factory… Continue reading Azure Data Factory Data Flow: Change Data Capture Architecture & Demo
MERGE command in Azure Synapse Analytics Dedicated SQL pool
In this article we will understand the features of MERGE T-SQL command and have a quick demo on Source table to Sync with target dimension table in Azure Synapse Analytics Dedicated SQL pool. MERGE T-SQL command for Azure Synapse Dedicated SQL pools is now Generally Available MERGE T-SQL feature: It combines INSERTs/UPDATEs/DELETEs into a single… Continue reading MERGE command in Azure Synapse Analytics Dedicated SQL pool
Azure Synapse Link for Azure SQL Database using Change Feed : Demo
In this article we will have a step-by-step demo on how to implement Azure Synapse Link for Azure SQL Database using Change Feed and load data into Azure Synapse Analytics. What is Azure Synapse Link for Azure SQL: ⏸️ Azure Synapse Link for SQL Extract data incrementally from transactional databases and Load it directly into… Continue reading Azure Synapse Link for Azure SQL Database using Change Feed : Demo
Azure Managed Identity Authentication for Azure SQL Database by Azure Data Factory
In this article we will have a step by step demo on how usingAzure Managed Identity, Azure Data Factory authenticates Azure SQL Database and Synapse Workspace authenticates Azure Synapse Analytics. Microsoft Recommendation on Passwordless Authentication Managed identities are the recommended authentication option for secure, passwordless connections between Azure resources. Azure Managed Identity Authentication: WHAT it… Continue reading Azure Managed Identity Authentication for Azure SQL Database by Azure Data Factory
Delta Lake’s Change Data Feed (CDF) Demo in Azure Databricks
In this article we will have a demo on implementing Delta Lake’s Change Data Feed (CDF) using Multi-hop or Medallion Architecture in Azure Databricks. Need for Change Data Feed (CDF): Simplify CDC: Change Data Feed (CDF) addresses these challenges we faces using Change Data Capture (CDC) like improving: Quality Control by tracking row level changes.… Continue reading Delta Lake’s Change Data Feed (CDF) Demo in Azure Databricks
Microsoft Azure Unified Data and Analytics Architecture
𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭 𝐀𝐳𝐮𝐫𝐞 𝐧𝐚𝐭𝐢𝐯𝐞 𝐮𝐧𝐢𝐟𝐢𝐞𝐝 𝐃𝐚𝐭𝐚 𝐚𝐧𝐝 𝐚𝐧𝐚𝐥𝐲𝐭𝐢𝐜𝐬 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦: 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 Let’s have a quick view on the sample use cases and architectural components of this Unified Data and Analytics platform using Azure native data services. This architecture consistent of all components From data source to staging blob storage via ingestion and Subsequently clean and transformation, Stores… Continue reading Microsoft Azure Unified Data and Analytics Architecture
Cast Transformation DEMO in Mapping Data flow of Azure Data Factory
Recently Cast transformation is introduced to Mapping Data Flow in Azure Data Factory and Azure Synapse Analytics. In this article, we will perform a quick demo on how to use this cast transformation in data flow. We will also explore every option available in Cast transformation Settings. How Cast Transformation helps Data Engineer? It allows… Continue reading Cast Transformation DEMO in Mapping Data flow of Azure Data Factory
How Microsoft Azure IoT helps XTO Energy to monitor field assets in real time
XTO Energy built an IoT Solution in Microsoft Azure to derive valuable insight from data generated by IoT devices ties on field assets i.e oil-well in the area of Permian Basin. XTO Energy placed existing sensors at the oil-well head to monitor key system parameters like temperatures, pressures, and flow rates. As a result, IoT… Continue reading How Microsoft Azure IoT helps XTO Energy to monitor field assets in real time
Why Meta selected Azure as a Strategic Cloud Provider to accelerate AI Research and Development?
As part of expansion of ongoing collaboration with Meta, Microsoft announced that Meta has selected Azure as a strategic cloud provider to help accelerate AI research and development and also scale PyTorch adoption on Azure. “We are excited to deepen our collaboration with Azure to advance Meta’s AI research, innovation, and open-source efforts in a… Continue reading Why Meta selected Azure as a Strategic Cloud Provider to accelerate AI Research and Development?
Azure Synapse Analytics Introduced Intelligent Cache for Apache Spark to improve performance of repeat queries
Microsoft recently introduced Intelligent Cache for Apache Spark engine in Azure Synapse Analytics as part of Azure Synapse Caching. This Intelligent Cache of Apache spark pool in synapse reduces the total cost of ownership (TCO) by improving performance for reading the remote file types in data lake: Parquet files: up to 65% on subsequent reads… Continue reading Azure Synapse Analytics Introduced Intelligent Cache for Apache Spark to improve performance of repeat queries