Microsoft Azure Unified Data and Analytics Architecture

๐Œ๐ข๐œ๐ซ๐จ๐ฌ๐จ๐Ÿ๐ญ ๐€๐ณ๐ฎ๐ซ๐ž ๐ง๐š๐ญ๐ข๐ฏ๐ž ๐ฎ๐ง๐ข๐Ÿ๐ข๐ž๐ ๐ƒ๐š๐ญ๐š ๐š๐ง๐ ๐š๐ง๐š๐ฅ๐ฒ๐ญ๐ข๐œ๐ฌ ๐ฉ๐ฅ๐š๐ญ๐Ÿ๐จ๐ซ๐ฆ: ๐€๐ซ๐œ๐ก๐ข๐ญ๐ž๐œ๐ญ๐ฎ๐ซ๐ž

Letโ€™s have a quick view on the sample use cases and architectural components of this Unified Data and Analytics platform using Azure native data services.

This architecture consistent of all components
  • From data source to staging blob storage via ingestion and
  • Subsequently clean and transformation,
  • Stores into enterprise cloud data ware house and
  • create semantic model out of data in Data warehouse and make it available for BI consumption.

Please refer the numbers on the components in the architecture diagram while viewing the Architectural Components below:

Microsoft Azure Unified Data and Analytics Architecture (Image credit: Microsoft)
๐“๐ฒ๐ฉ๐ข๐œ๐š๐ฅ ๐”๐ฌ๐ž ๐‚๐š๐ฌ๐ž๐ฌ:
  • Using Microsoft Azure native data services, Organization can build data warehouse as single source of truth while integrating relational data with unstructured data.
  • Data analyst can use Power BI to visualize data after Semantic modelling being applied.
๐€๐ซ๐œ๐ก๐ข๐ญ๐ž๐œ๐ญ๐ฎ๐ซ๐ž ๐‚๐จ๐ฆ๐ฉ๐จ๐ง๐ž๐ง๐ญ๐ฌ:
  1. ๐‘ซ๐’‚๐’•๐’‚ ๐’”๐’๐’–๐’“๐’„๐’†: SQL Server on-premises, Oracle on-premises, Azure SQL Database, Azure table storage, Cosmos DB
  2. ๐‘จ๐’›๐’–๐’“๐’† ๐‘ฉ๐’๐’๐’ƒ ๐’”๐’•๐’๐’“๐’‚๐’ˆ๐’†: all source data to be staged here. Azure Data Factory can be used to move the data from source to this staging blob storage.
  3. ๐‘จ๐’›๐’–๐’“๐’† ๐‘ซ๐’‚๐’•๐’‚ ๐‘ญ๐’‚๐’„๐’•๐’๐’“๐’š: ADF incrementally loads the data from Blob storage, cleansed and transformed and storesย  into staging tables in Azure Synapse Analytics.ย  PolyBase helps parallelize the process for large datasets.
  4. ๐‘จ๐’›๐’–๐’“๐’† ๐‘บ๐’š๐’๐’‚๐’‘๐’”๐’†: As a distributed system for storing and analyzing large datasets, itโ€™s massive parallel processing (MPP) makes it suitable for running high-performance analytics.
  5. ๐‘จ๐’๐’‚๐’๐’š๐’”๐’Š๐’” ๐‘บ๐’†๐’“๐’—๐’Š๐’„๐’†๐’”: provides a semantic model for data.
  6. ๐‘ท๐’๐’˜๐’†๐’“ ๐‘ฉ๐‘ฐ : a business analytics tools to analyze data and share insights by querying a semantic model stored in Analysis Services or Azure Synapse directly.
  7. ๐‘จ๐’›๐’–๐’“๐’† ๐‘จ๐’„๐’•๐’Š๐’—๐’† ๐‘ซ๐’Š๐’“๐’†๐’„๐’•๐’๐’“๐’š (๐‘จ๐’›๐’–๐’“๐’† ๐‘จ๐‘ซ): it authenticates users who connect to the Analysis Services server through Power BI.

This architectural diagram was to implement data ingestion and transformation and itsโ€™s data ware housing for BI consumption using all Azure native services. Thanks for reading. Please feel free to comment below in case of any query.

More article on Azure Data factory and synapse:

Cast Transformation DEMO in Mapping Data flow of Azure Data Factory: https://sarnendude.com/cast-transformation-in-mapping-data-flow-of-azure-data-factory-synapse-analytics/

How to create and use Flowlet transformation in Azure Data Factory and Azure Synapse pipeline: https://sarnendude.com/how-to-create-and-use-flowlet-transformation-in-azure-data-factory-and-azure-synapse-pipeline/

Leave a Reply