In this article we will use Azure Data Factory Copy Activity to copy the JSON data from Azure Data Lake Gen 2 to Azure Synapse Analytics SQL Pool i.e. copy from Hierarchical source to tabular sink in the form of Hierarchical Structures to Relational format. When to use ADF Copy Activity & Data flow? ADF… Continue reading Azure Data Factory Copy Activity: Copy Hierarchical JSON data to Azure Synapse Analytics SQL Pool
Azure Synapse Analytics: Map Data tool Demo Step by Step
Microsoft recently announced ‘Map Data’ tool, a new feature of Azure Synapse Analytics. Update: Map Data tool in Synapse Analytics is now Generally Available as on Jul 12 2022. In this article we will demonstrate the step by step on how to use this new feature in Azure Synapse Analytics by mapping data between Azure… Continue reading Azure Synapse Analytics: Map Data tool Demo Step by Step
Azure Solutions Architect Expert Certification Redesigned: Exam AZ-305 is live now
Exam AZ-305: Designing Microsoft Azure Infrastructure Solutions is LIVE Now: Now this is Exam AZ-305 is one of prerequisite to earn Azure Solutions Architect Expert Certification. How Azure Solutions Architect Expert Certification paths has been evolved: PAST to FUTURE -How it is evolving? PAST: (AZ-300 + AZ-301) – Already retired. PRESENT: (AZ-303 + AZ-304) –… Continue reading Azure Solutions Architect Expert Certification Redesigned: Exam AZ-305 is live now
Azure Synapse Lake Database: WHY & HOW to get started?
In this article we will discuss WHY we need lake database in Azure Synapse i.e. what challenges it solves and HOW it solves these challenges and HOW to get started with lake database. Currently this Azure Synapse Lake database is in public preview. The lake database in Azure Synapse Analytics enables customers to bring together… Continue reading Azure Synapse Lake Database: WHY & HOW to get started?
Microsoft Azure Data Factory (ADF) Managed Virtual Network is now Generally Available
Microsoft Azure Data Factory (ADF) Managed Virtual Network is now Generally Available: By using Managed Virtual Network (VNet), ADF provides more security to data integration and data movement by creating Azure Integration Runtime (IR) with managed VNet i.e. compute environment with in VNet. using Private Endpoints to securely connect to supported data stores Thus it… Continue reading Microsoft Azure Data Factory (ADF) Managed Virtual Network is now Generally Available
Azure Data Factory mapping data flow: ‘Source’ partitioning on Azure SQL Database
Azure Data Factory & Azure Synapse Pipelines both provides mapping data flows for the data transformation. Each data flow should contain at least one ‘source transformation‘ which configures the data source for the data flow. We can configure a source transformation by adding a data source using ‘Add Source’ box in the data flow canvas.… Continue reading Azure Data Factory mapping data flow: ‘Source’ partitioning on Azure SQL Database
IoT Architecture: End-to-End IoT Analytics & Machine Learning with Microsoft Azure Data/AI services & Databricks
Sample use case: Collect sensor data from the IOT asset (like turbine) & Build insights on short-term optimization as well as longer term maintenance costs -by ingesting data into the cloud for storage and analytics. IoT Architecture Main Components: (please map below sequence number with numbers in diagram ) Azure IoT Hub (recommended) or Event… Continue reading IoT Architecture: End-to-End IoT Analytics & Machine Learning with Microsoft Azure Data/AI services & Databricks
Microsoft launched new Azure Availability Zone in its Central India data center region (Pune):
What does this mean to Microsoft Cloud customers in India? With uptime of up to 99.99%, this will help Microsoft customers in India to deploy their infrastructure and applications across the data centers’ locations for high availability, resiliency & security. For their cloud application, now customers in India will get below additional protection/benefit in terms… Continue reading Microsoft launched new Azure Availability Zone in its Central India data center region (Pune):
Databricks SQL Analytics introduction
Databricks SQL Analytics – Quick introduction: Databricks SQL is now Generally Available (GA) providing a new generation of analytics & data applications, running directly on the data lake with data warehousing capabilities & SQL support on the Databricks Lakehouse Platform. Databricks SQL (DB SQL): allows us to operate on a multi-cloud lakehouse architecture provides thousands of optimizations… Continue reading Databricks SQL Analytics introduction
Azure Stream Analytics job now has new output option: Azure PostgreSQL (Preview)
Quick Update: Microsoft introduced new output option of Azure Stream Analytics job to Azure PostgreSQL (Preview): This new output option to Azure PostgreSQL allows customer to write output streaming/analytics data into relational database. With this new output, Azure Stream Analytics jobs write to Azure PostgreSQL Single Server, Flexible Server (Preview) and Hyperscale (Citus), supporting high throughput… Continue reading Azure Stream Analytics job now has new output option: Azure PostgreSQL (Preview)