Data Architecture
Last updated
Last updated
Data Mesh is a modern architectural approach to building data infrastructure that aims to make data ownership and governance decentralized and distributed. In this approach, data is organized into domain-specific data products that are owned and managed by the individual domain teams. Each data product has its own schema and is responsible for its own data quality, availability, and accessibility.
Imagine a large e-commerce company that has multiple domain teams such as product, marketing, finance, and logistics. In a data mesh architecture, each domain team would be responsible for creating and managing their own data products such as product catalog, customer data, marketing campaigns, financial transactions, and shipping records. Each data product would have its own schema and would be published to a centralized data catalog for easy discovery and access.
Data Vault is a modeling technique for designing data warehouses that provides flexibility, scalability, and resilience to changes in the data sources. In this approach, data is modeled as a set of hubs, links, and satellites. The hubs represent the business entities, the links represent the relationships between them, and the satellites represent the attributes of the entities and relationships over time.
Consider a healthcare company that wants to analyze patient data from multiple sources such as electronic health records, insurance claims, and clinical trials. In a data vault approach, the data would be modeled as a set of hubs, links, and satellites. For example, the patient hub would contain a unique identifier for each patient, the claim link would connect the patient hub to the insurance claim information, and the diagnosis satellite would contain information about the patient's medical condition.
Kimball is a methodology for designing and building data warehouses that emphasizes business requirements and user needs. In this approach, data is organized into dimensionally modeled data marts that are optimized for specific business areas. The methodology is based on four key steps: requirements gathering, dimensional modeling, physical design, and implementation.
Imagine a telecom company that wants to analyze their customer data such as call logs, text messages, and data usage. In a Kimball approach, the company would start by gathering requirements from business users such as marketing and customer support. Then, they would create dimensionally modeled data marts for each business area such as customer behavior, product usage, and customer service.
Inmon is another methodology for designing and building data warehouses that focuses on the integration of data from disparate sources. In this approach, data is first integrated into a central data warehouse and then transformed into a set of subject-oriented data marts that are optimized for specific business areas. The methodology is based on three key steps: data integration, enterprise data warehousing, and data mart creation.
Consider a financial services company that wants to analyze their data from multiple sources such as banking transactions, credit card purchases, and loan applications. In an Inmon approach, the company would first integrate the data into a central data warehouse. Then, they would transform the data into subject-oriented data marts such as banking, credit card, and loans. This approach would provide a unified view of the company's data and enable cross-functional analysis.
