Kristyna Ferris on Data Warehousing with Microsoft Fabric at SQL Saturday Chicago

Day - 8 | Microsoft Fabric Series Medallion Architecture. This is where everything starts to make sense. Lakehouse. Data Warehouse. OneLake. They don't work in isolation. They work together — inside a framework called Medallion Architecture. And it's simpler than it sounds. Three layers. Three jobs. 🥉 Bronze — data lands here first. Raw. Untouched. Exactly as it came. 🥈 Silver — data gets cleaned here. Duplicates removed. Nulls fixed. Standardized. 🥇 Gold — data is ready here. Modeled. Aggregated. Trusted. Every layer has one job. Every layer feeds the next. ✨ The best part? It all happens inside Microsoft Fabric. On the same foundation — OneLake. Bronze to Silver to Gold. Raw to ready. In one platform. 🤔 Which layer do you think is the most important — Bronze, Silver or Gold? Comment below. Next up: Data Factory — how data actually gets into Microsoft Fabric. #MicrosoftFabric #PowerBI #DataEngineering #DataAnalytics #MedallionArchitecture #DataArchitecture #ModernDataStack

Two Small Microsoft Fabric Updates That Solve Big Data Problems In Lakehouse architectures, pipelines constantly load data and update schemas. But one common challenge is: Is the analytics layer ready for reporting? Two new Microsoft Fabric pipeline activities help solve this: 1️⃣ Refresh SQL Endpoint Each Fabric Lakehouse has a **SQL Analytics Endpoint** used by tools like **Microsoft Power BI** to query Delta tables. After schema or table changes, the SQL endpoint metadata may not update immediately. **New Feature:** A native pipeline activity to **refresh the SQL endpoint metadata**. **Benefit** * BI tools see the latest schema instantly * No need for manual refresh, APIs, or notebooks 2️⃣ Lakehouse Maintenance Delta tables can accumulate **many small files** from frequent data updates, which slows queries. **New Feature:** A Lakehouse Maintenance activity that runs operations like: * OPTIMIZE * VACUUM * File compaction **Benefit** * Faster queries * Reduced storage fragmentation * Healthier Delta tables Why It Matters These updates make Microsoft Fabric pipelines more powerful by: * Reducing reliance on scripts and notebooks * Improving BI reliability * Optimizing Lakehouse performance Sometimes small platform updates bring big operational improvements.

The energy around SQL Pools in Microsoft Fabric at Microsoft Fabric Community Conference has been incredible. So we decided to do what data people love most… Spin it up and test it LIVE. In this video we walk through the brand-new SQL Pools capability and show exactly how it works in practice. No slides. No theory. Just a live demo of how you can stand up a pool and start thinking about how it can accelerate data engineering workloads inside Fabric. If you're working with large scale pipelines, heavy transformations, or parallel compute patterns, this feature is going to get your attention. Watch the demo here: https://lnkd.in/g-n6kgCf What we cover in the video: - What SQL Pools are and why Microsoft introduced them - How to create one inside Fabric - When to use SQL Pools vs other Fabric compute options - Why this is a big step forward for Fabric data engineering Huge announcements like this are why the Fabric ecosystem keeps getting more exciting. The data gods are cooking. #MicrosoftFabric #FabCon #DataEngineering #SQL #Analytics #PowerBI #DataPlatform #FabConSQLCon2026

The biggest mistake commonly see in Microsoft Fabric isn’t pipelines, it’s a Lakehouse design. Most teams build Medallion Architecture like this 1. One Lakehouse ➡️ Three folders: Bronze Silver Gold Looks neat on Paper and Board, but Breaks governance. Here is the issue ? In Microsoft Fabric ,Access is controlled at the Lakehouse level (via SQL Analytics Endpoint). 👉 One Lakehouse = One boundary 👉 Your Bronze PII + Gold BI data = same boundary. Now Governance becomes… folder names 😬 ✅ What works in real production setups 1 Lakehouse per layer or Separate Gold with Warehouse 🔹 Bronze → Raw, append-only, no human access 🔹 Silver→ Cleaned, structured, owned by Data Engineering 🔹 Gold → Business-ready, read-only for BI 👉 3 Lakehouses = 3 security boundaries 👉 Governance becomes structural, not cosmetic ⚡ Two Common Misses ✔️ Use OneLake Shortcuts → No data duplication across layers → Single physical copy, multiple logical views ✔️ Enable V-Order on Gold → Required for Power BI Direct Lake performance → Missing it = silent fallback to DirectQuery = slow dashboards 🧠 Key Takeaway Fabric is not storage with a UI. It’s an architecture. 👉 One copy of data 👉 Three governed layers 👉 Clean lineage 👉 In-memory BI performance #MicrosoftFabric #DataEngineering

🤔 Question for the Microsoft Fabric community As Microsoft Fabric evolves rapidly, I’m curious how others are thinking about data architecture patterns in the new ecosystem. For years, many of us followed the traditional Bronze 🥉 → Silver 🥈 → Gold 🥇 (Medallion) architecture: • Bronze – Raw ingestion • Silver – Cleaned and standardized data • Gold – Aggregated / business-ready datasets This made perfect sense in earlier lakehouse implementations. But with Fabric capabilities like OneLake Lakehouse Warehouse Direct Lake Delta Lake tables and Materialized Views I’m wondering if the architecture can be simplified. For example: * If we already have clean curated tables (Silver)/Operational Data Store * And we can create Materialized Views for aggregations and performance 👉 Do we still need to physically maintain a Gold layer? Could Materialized Views + Semantic Models effectively replace the traditional Gold layer in many scenarios? In other words, instead of: Bronze → Silver → Gold Are we moving toward something like: Raw → Curated → Serving (Materialized Views / Semantic Models) I’d love to hear how others in the **Fabric community are approaching this. Are you: 1️⃣ Still implementing full Medallion layers? 2️⃣ Simplifying the architecture? 3️⃣ Using Materialized Views instead of maintaining Gold tables? Curious to learn from the community’s real-world experiences. #MicrosoftFabric #DataEngineering #Lakehouse #DataArchitecture #OneLake #PowerBI

Most people are learning Microsoft Fabric the wrong way!.. They focus on tools… 👉 Instead of understanding the architecture. 💡 If you understand THIS flow, you understand Fabric: 📥 Data Sources → ⚙️ Pipelines → 🌊 OneLake → 🧱 Lakehouse / Warehouse → ⚡ Direct Lake → 📊 Power BI 🔥 The biggest shift happening right now: ❌ Multiple tools (ADF + SQL + Storage + BI) ✅ ONE unified platform (Fabric) 💡 What makes Fabric powerful isn’t just features… It’s how everything connects seamlessly. 👉 No unnecessary data movement 👉 No duplicated storage 👉 Faster time to insight ⚠️ And here’s the mistake I see in real projects: Teams are still building old architectures inside new platforms 🚀 Reality: Fabric is not just an upgrade… 👉 It’s a complete mindset shift 💬 Curious — Are you still using traditional architecture, or already building in Fabric? Comment FABRIC 👇 — Sreeni 🚀📊 #MicrosoftFabric #PowerBI #DataEngineering #DataArchitecture #Azure #Lakehouse

𝐇𝐨𝐰 𝐓𝐨𝐩 1% 𝐓𝐞𝐚𝐦𝐬 𝐔𝐬𝐞 𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭 𝐅𝐚𝐛𝐫𝐢𝐜 Microsoft Fabric gives you a powerful unified platform: • OneLake • Multiple compute engines (Spark, SQL, KQL, BI) • Integrated analytics experience But it does not give you ownership, governance, or architecture by default. High-performing teams succeed because they deliberately add structure where Fabric stays flexible: ✔ Define clear data domains and ownership ✔ Control how OneLake is organized and accessed ✔ Separate workloads by convention (Spark → transform, Warehouse → serve, BI → consume) ✔ Use Delta Lake for contracts—not just storage ✔ Balance certified datasets with self-service BI ✔ Maintain Dev/Test/Prod discipline ✔ Monitor and plan capacity usage carefully ✔ Combine Purview with strong governance processes ✔ Use real-time analytics selectively ✔ Design for interoperability beyond Fabric Fabric reduces friction. It does not reduce responsibility. Average teams centralize data in Fabric. Top 1% teams build clarity, discipline, and scalable operating models on top of it. More concise implementation details are available in the presentation attached below.

Microsoft Fabric is not just another data tool; it’s transforming our approach to data engineering. In the past, building a data platform required navigating through multiple tools: - Azure Data Factory for pipelines - Databricks for transformations - Synapse for warehousing - Power BI for reporting While everything was interconnected, it still felt disjointed. Recently, I worked on a Fabric-based Lakehouse project, and for the first time, everything felt unified: - Ingestion with pipelines - Storage in OneLake - Transformation using notebooks - Reporting directly in Power BI All in one place. What stood out to me was how implementing medallion architecture became much simpler: - Bronze → raw data - Silver → cleaned data - Gold → business-ready data This approach eliminates unnecessary complexity. My takeaway is that Fabric is not about replacing tools - it’s about reducing friction in data engineering. While it is still evolving, it represents a significant shift that deserves attention. Are you exploring Fabric yet? #MicrosoftFabric #DataEngineering #Azure #Lakehouse #PowerBI #DataAnalytics

Dataflow Gen2 is a powerful low-code data transformation tool that helps you easily prepare and move data for analytics. Built on Power Query, it enables users to create scalable and reusable ETL pipelines without heavy coding. With seamless integration into Microsoft Fabric and direct loading into Microsoft OneLake, Dataflow Gen2 simplifies data preparation across your organization. ✅ Low-code data transformation ✅ Reusable data pipelines ✅ Seamless Fabric integration ✅ Scalable data preparation #MicrosoftFabric #DataEngineering #DataflowGen2 #PowerBI #Analytics #DataTransformation

Hello everyone, Day 4/30 — Microsoft Fabric Series Today's topic → Lakehouse 🏡 When I first heard "Lakehouse" I thought- is this a Data Lake or a Data Warehouse? The answer is both. And that's exactly the point. The old problem 👇 Data Lake → great for storage, painful to query. Data Warehouse → great to query, expensive to store. So most teams ended up with both. And a painful pipeline moving data between them. Lakehouse solves this 👇 ✅ Flexibility of a Data Lake → store any data, any format ✅ Performance of a Data Warehouse → query with SQL, fast and reliable ✅ Built on Delta Parquet → open, versioned, ACID compliant ✅ Sits directly on OneLake → no copying, no moving data How I use it in real projects 👇 This is where Medallion Architecture lives inside Fabric. 🥉Bronze Lakehouse → raw data as-is 🥈 Silver Lakehouse → cleaned and validated 🥇 Gold Lakehouse → business ready data Three Lakehouses. One OneLake. Clean separation. Zero duplication. Before Fabric I maintained separate Data Lake and Synapse environments. Now everything lives in one Lakehouse. No duplicate storage. No sync headaches. #MicrosoftFabric #Lakehouse #DataEngineering #Azure #30DayChallenge #DeltaLake #MedallionArchitecture #CloudData