𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭 𝐀𝐳𝐮𝐫𝐞 𝐧𝐚𝐭𝐢𝐯𝐞 𝐮𝐧𝐢𝐟𝐢𝐞𝐝 𝐃𝐚𝐭𝐚 𝐚𝐧𝐝 𝐚𝐧𝐚𝐥𝐲𝐭𝐢𝐜𝐬 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦: 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞
Let’s have a quick view on the sample use cases and architectural components of this Unified Data and Analytics platform using Azure native data services.
This architecture consistent of all components
- From data source to staging blob storage via ingestion and
- Subsequently clean and transformation,
- Stores into enterprise cloud data ware house and
- create semantic model out of data in Data warehouse and make it available for BI consumption.
Please refer the numbers on the components in the architecture diagram while viewing the Architectural Components below:
𝐓𝐲𝐩𝐢𝐜𝐚𝐥 𝐔𝐬𝐞 𝐂𝐚𝐬𝐞𝐬:
- Using Microsoft Azure native data services, Organization can build data warehouse as single source of truth while integrating relational data with unstructured data.
- Data analyst can use Power BI to visualize data after Semantic modelling being applied.
𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐂𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭𝐬:
- 𝑫𝒂𝒕𝒂 𝒔𝒐𝒖𝒓𝒄𝒆: SQL Server on-premises, Oracle on-premises, Azure SQL Database, Azure table storage, Cosmos DB
- 𝑨𝒛𝒖𝒓𝒆 𝑩𝒍𝒐𝒃 𝒔𝒕𝒐𝒓𝒂𝒈𝒆: all source data to be staged here. Azure Data Factory can be used to move the data from source to this staging blob storage.
- 𝑨𝒛𝒖𝒓𝒆 𝑫𝒂𝒕𝒂 𝑭𝒂𝒄𝒕𝒐𝒓𝒚: ADF incrementally loads the data from Blob storage, cleansed and transformed and stores into staging tables in Azure Synapse Analytics. PolyBase helps parallelize the process for large datasets.
- 𝑨𝒛𝒖𝒓𝒆 𝑺𝒚𝒏𝒂𝒑𝒔𝒆: As a distributed system for storing and analyzing large datasets, it’s massive parallel processing (MPP) makes it suitable for running high-performance analytics.
- 𝑨𝒏𝒂𝒍𝒚𝒔𝒊𝒔 𝑺𝒆𝒓𝒗𝒊𝒄𝒆𝒔: provides a semantic model for data.
- 𝑷𝒐𝒘𝒆𝒓 𝑩𝑰 : a business analytics tools to analyze data and share insights by querying a semantic model stored in Analysis Services or Azure Synapse directly.
- 𝑨𝒛𝒖𝒓𝒆 𝑨𝒄𝒕𝒊𝒗𝒆 𝑫𝒊𝒓𝒆𝒄𝒕𝒐𝒓𝒚 (𝑨𝒛𝒖𝒓𝒆 𝑨𝑫): it authenticates users who connect to the Analysis Services server through Power BI.
This architectural diagram was to implement data ingestion and transformation and its’s data ware housing for BI consumption using all Azure native services. Thanks for reading. Please feel free to comment below in case of any query.
More article on Azure Data factory and synapse:
Cast Transformation DEMO in Mapping Data flow of Azure Data Factory: https://sarnendude.com/cast-transformation-in-mapping-data-flow-of-azure-data-factory-synapse-analytics/
How to create and use Flowlet transformation in Azure Data Factory and Azure Synapse pipeline: https://sarnendude.com/how-to-create-and-use-flowlet-transformation-in-azure-data-factory-and-azure-synapse-pipeline/