Speaking

Microsoft Fabric

Medallion Architecture Without Over-Engineering

Bronze, Silver and Gold as responsibilities rather than mandatory boxes: when all three layers help, when fewer are better, and how Delta, incremental processing, file design and data quality fit in.

Status
Proposed
Type
Talk
Formats
Webinar · Workshop · Internal session
Level
Beginner
Duration
45 minutes (talk) · 2–3 hours (workshop)

This is a proposed session topic. No public event is currently scheduled.

Who it is for

  • Data engineers starting or reviewing a lakehouse platform
  • Teams that inherited three layers and are unsure what each one is for

What attendees will learn

  • What Bronze, Silver and Gold are responsible for
  • How to recognise when a layer adds cost without adding value
  • Why Delta tables make layered processing practical, and where they do not help
  • How incremental processing, partitioning and file sizing change across layers
  • How data quality checks evolve from Bronze to Gold

Session overview

Medallion Architecture is one of the most useful patterns in modern data platforms, and one of the most over-applied. This session treats it as what it is: a way to give each processing stage a clear purpose and boundary. It is not a requirement to copy every table three times. It is based on Medallion Architecture Explained and turns that material into a guided walkthrough with examples.

  1. SourcesApplications, files, APIs, databases
  2. BronzeLand the data as received
  3. SilverClean, validate, conform
  4. GoldShape for a business use
  5. ConsumersReports, models, APIs
The flow the session starts from, before asking which layers each workload really needs.

Outline

  1. What each layer is for: source fidelity, conformance, and serving.
  2. Do you need three layers? Simple and complex workloads compared, and the cost of every extra layer.
  3. Delta tables: transactions, MERGE, schema evolution and history, plus what they do not solve.
  4. Incremental processing across layers: from Bronze appends to targeted Gold refreshes.
  5. Partitioning and small files: different choices per layer.
  6. Data quality as a progression: “did we receive it?”, “is it valid?”, “is it right for the business?”
  7. Common mistakes, and how to recover from them.

Adapting the session

  • Talk or webinar: concepts and decision criteria, with diagrams.
  • Workshop: attendees decide the layer design for three sample workloads and implement one end to end in Fabric notebooks.

Tags

  • Medallion Architecture
  • Bronze
  • Silver
  • Gold
  • Delta Lake
  • Incremental Loads
  • File Sizing
  • Microsoft Fabric