Peter Vlekken

🚀 We’ve Envisioned the Future of AI — and You’re Already Building It. But there’s one thing standing between hype and real-world impact: the DATA that is powering your AI.💡 That’s why we created the Denodo GenAI Manifesto: a bold vision for how trusted, real-time, and explainable data must shape the next era of AI. 📜🤖 Too often, AI is built on brittle, black-box foundations. If we want #AI that delivers real outcomes — not hallucinations — we need data that’s: ✅ Governed 🔍 Transparent 🧭 Context-rich ⚡ Instantly accessible 📢 The Manifesto is a call for business and tech leaders to rethink how we integrate, govern, and activate data in the age of #GenAI. 👉 Read it. Share it. Challenge it. The next decade of innovation depends on how we build today: https://okt.to/x2Sf6w #GenAIManifesto #DataStrategy #RealTimeData #AIReadyData

4

𝗪𝗵𝗲𝗻 (𝗮𝗻𝗱 𝘄𝗵𝗲𝗻 𝗻𝗼𝘁) 𝘁𝗼 𝘂𝘀𝗲 𝗗𝗮𝘁𝗮𝗯𝗿𝗶𝗰𝗸𝘀? Most teams don’t fail with Databricks because the platform is bad. They fail because Databricks was never the right fit. In this guest post, Jakub Lasak shares practical lessons from real production environments, covering: - When Databricks genuinely makes sense. - When it becomes overkill . - Common adoption mistakes teams make. - How to think about cost realistically. One of the more practical takes on lakehouse adoption and platform trade-offs by a real Databricks data engineer. Don’t forget to subscribe to Jakub's substack dataengineerwiki for more Databricks and data engineering content. https://lnkd.in/guC9RcE6

39

Data is the lifeblood of your organization, and MDM is its heart. 🫀 Many companies treat data quality as an "IT problem." That’s a mistake. As this graphic shows: Data quality is everyone’s responsibility. Why is this critical? • Production: Inaccurate inventory = costly downtime. • HR: Outdated records = workforce chaos. • R&D: Inconsistent specs = flawed products. • Governance: Poor data ethics = legal risks and AI bias. The rule is simple: Garbage In, Garbage Out. If we pump "polluted" data into the MDM hub, the entire organization suffers from informational blockages. Let’s build a culture where everyone acts as a steward of the data they touch. Awareness is the foundation of success. 🚀 #DataQuality #MDM #MasterData #DataGovernance #BusinessIntelligence #DataCulture #DataSteward #DataStewardship

10

We help clients think more broadly about Data UI, and put it into action 🚀 From agentic analytics to data apps replacing SaaS sprawl, the leading data teams are enabling organisations to engage with data differently… #data #ai Sigma AND Digital

18

The era of the "Static Dashboard" is ending. The era of the "Data Agent" has begun. 📉➡️🤖 I’ve been experimenting with the Looker MCP (Model Context Protocol) Server, and it’s arguably the most important shift in BI since the semantic layer itself. For years, the biggest blocker to using GenAI on enterprise data was Trust. If you ask an LLM to "query my sales data," it usually tries to guess the SQL schema. It hallucinates. It fails. Looker + MCP changes the physics of this problem. Instead of the AI guessing the data structure, MCP gives agents (like Claude Desktop or Gemini) a direct, standardized plug into Looker’s Semantic Layer. 1️⃣ The Agent asks for a concept ("Gross Margin by Region"). 2️⃣ Looker calculates it using governed logic (no hallucinations). 3️⃣ The Agent receives the trusted data and takes action. We are no longer just building dashboards for executives to stare at. We are building the "Sense-Making API" for the AI workforce. If you are a data leader, your Semantic Layer just became your most valuable AI asset. #Looker #GoogleCloud #MCP #AI #DataEngineering #GenAI Kartik Bhanot Neeti Sukhtankar Nitin Parab Sushant Kashyap Ramanpreet Singh Arora Vishal Lakhina

28

Today's chapter of #databard: Is AI Deterministic enough? As data professionals, we like it when a process returns the same outcome every time, given the same data and variables. While Generative AI can generate code, its intentional, creative nature seems at odds with determinism. Does this mean that using AI to support our work just doesn't work at scale to deliver consistent, reliable results? In this video, I share my thoughts on the topic. #genai #ai #ssmscopilot #deterministic https://lnkd.in/gdVC2aDS

6

I just completed Data Management at Scale, 2nd Edition by Piethein Strengholt And it’s a strong reminder that modern data is no longer about centralization, but about scalability, ownership, and value creation. Here are some key takeaways : - Moving from centralized data warehouses to distributed architectures (Data Mesh, Data Fabric) - Designing domain-oriented data products with clear ownership and accountability - Implementing scalable data pipelines and cloud-native architectures (Azure, Lakehouse, APIs) - Strengthening data governance frameworks, data quality, and data security - Leveraging metadata management and data catalogs to enable self-service analytics - Aligning data initiatives with business value, data-driven decision-making, and operational efficiency What stood out most is the shift toward treating data as a product, not just an asset, with a strong focus on stakeholder collaboration, data ownership, and cross-functional alignment. In today’s market, where organizations are looking for profiles skilled in Data Architecture, Data Governance, Cloud Data Platforms, ETL/ELT, SQL, APIs, and Agile delivery, this mindset is no longer optional, it’s essential. I help organizations architect and scale modern data platforms, enabling robust data governance, reliable data pipelines, and high-quality data products that drive actionable insights and business performance. #DataArchitecture #DataGovernance #DataMesh #DataFabric #CloudComputing #Azure #DataEngineering #BusinessAnalysis #DataAnalytics #Agile #DataStrategy

29

Our client asked: "How do we know this number is right?" We realized: We didn't. The data pipeline worked. Reports generated. But we had no systematic way to validate if the numbers were actually correct. So we built a testing framework. What we implemented: Layer 1: Source Data Tests • Row counts match API pulls (did we get everything?) • Required fields are populated (no unexpected nulls) • Date ranges make sense (no future dates, no dates from 1970) Layer 2: Transformation Tests • Aggregations are additive (sum of parts = total) • Percentages fall within 0-100% range • Joins don't create duplicates unexpectedly Layer 3: Business Logic Tests • Revenue > 0 for active customers • Metrics fall within historical ranges (catch outliers) • Related metrics move together (if traffic drops, leads should drop) Now when a client asks "How do we know this is right?" we show them the tests. Still building, still learning, still finding edge cases. _______________ Comment TEST if you're building data quality frameworks PS - I sit and sip my favorite Rishi Tea & Botanicals drink after I build those data tests...

14

1 Comment

Three weeks ago, I gave my first conference talk at Big Data Conference Europe. My session was about building architectures that fit your needs instead of your vendors. I focused on the architecture components and it's requirements instead of technologies and covered: • When Databricks makes sense, and when dbt + DuckDB can be a better fit • A hands-on walkthrough of data platform architectures on Azure • How composable building blocks improve evolvability and cost efficiency If you couldn’t attend, the recording is now available on YouTube: https://lnkd.in/eeqR6_st #dataengineering #bigdataconference #dataminded

49

2 Comments

From Deterministic to Agentic ... The Real Fear Isn’t the Algorithm!! ‼️ Even deterministic (workflow) AI was tough to operationalize‼️ Clear inputs, fixed rules, measurable outcomes : and still, 90% of POCs never reached production. Now we’re moving toward agentic systems; AI that doesn’t just predict but plans! decides! and acts! That’s not a workflow anymore. That’s behavior. And that’s why it scares CTOs. Because the challenge ahead isn’t better algorithms!! It’s #governance, #guardrails, and #monitoring! The next generation of AI architecture won’t be defined by model accuracy. It’ll be defined by how well we can observe, constrain, and align autonomous systems with business intent! ❣️ And this isn’t my pessimistic side speaking... it’s the realistic one. So if #GenAI and AI agents is your niche, focus on #Governance. These are the skills the world needs right now! #agenticai #databricks

5

A few weeks ago, we launched a project to help teams build AI/BI dashboards with Databricks One. Today, we are taking the next step: Infusion Mode for the Dashboard Design & Insight Lab (name of our project)! Why? Because great dashboard design is not just about layout: it is also about style, typography, color harmony, and visual consistency. That's why we built Infusion Mode: style your AI/BI dashboard in 3 clicks 🎨 How it works (3 options): 1️⃣ Upload a dashboard you love → Agent analyzes its style and applies it to your dashboard automatically 2️⃣ Describe your style → "Clean enterprise," "bold and creative" → Agent generates and applies a visual theme instantly 3️⃣ Upload your company logo → Agent builds a complete branded visual theme and applies it With the Infusion Mode, we handle the aesthetic. You handle the strategy. Try Infusion Mode: https://lnkd.in/dSxavYjj Want to discuss or contribute? Ping me! Stay tuned for our next feature release 🚀 #Databricks #AIBIDashboard #DashboardDesign #BusinessIntelligence #DataAnalytics

109

3 Comments

Exceptional alpha coming from Eduardo Ariño de la Rubia in this repo. While AI might be handling more of the execution in the future, problem framing, systems thinking, and stakeholder alignment will likely still be in the hands of humans for some time to come. This resource is well worth checking out!

2

Modern data space = Organized chaos 🎪 Enterprise data stack can't even fit on one slide. Before one tool matures, two new ready to replace(shiny object syndrome). Every year new migration. Metadata, Lineage, catalog, quality, products… every tools promises it all( wait.. business problems?). So is this really a technology problem? or is it cultural maturity? architectural design gaps? Lack of discipline? maybe all 🤷 👀 I think the success edge comes from a clear long term data vision board(not sprint or month), right architecture, and the discipline to solve business problems. 👉 what do you think? #DataStrategy #DataGovernance #EnterpriseData #DataCulture

22

2 Comments

Yesterday, during a discussion on Snowflake internals, I realized that I had misunderstood a core concept: micro-partitions are immutable. At first, I made this assumption in the heat of the moment, but I turned out to be an interesting discussion further. This video documents the why behind Snowflake’s design and the how of what actually happens under the hood. In this video, I explain: 👉🏻 Why Snowflake micro-partitions are immutable by design 👉🏻 How updates and deletes really work without mutating existing partitions 👉🏻 The critical role metadata plays in query performance and partition pruning 👉🏻 What happens to logically deleted micro-partitions 👉🏻 The complexity Snowflake abstracts away so developers do not have to manage it themselves This was a genuine learning moment for me, and I wanted to share it openly in case others are thinking about Snowflake storage the same way I was. If you work with Snowflake or are preparing for SnowPro certifications, this mental model is important. The video is under 10 minutes and focuses purely on fundamentals, not surface-level features. I would appreciate feedback, corrections, or alternative perspectives from practitioners who work deeply with Snowflake internals. #Snowflake, #DataEngineering, #SnowPro, #CloudData, #DataArchitecture, #Metadata, #LearningInPublic, #SystemsThinking

21

I was pleasantly surprised with this result from NotebookLM on my first try. One of the reasons I wanted to try something like this is because most AI-generated content still reads like… AI — too polished, too structured, and not how people actually talk. I see a lot of that type of content on LinkedIn. I use these tools too — all the time — and they’re incredibly helpful for speeding up certain workflows. But when it comes to writing posts, there's a recognizable feel to AI-generated text. You can spot it instantly… and I usually scroll right past it. This experiment felt like it broke that mold — at least a little. Pause and take a listen. Curious what you think. The output still has some stiffness, but it’s more natural and engaging than your average LLM response. The conversational format brings a bit more personality and makes the ideas easier to connect with.

7

The journey of a data platform is more than just technology—it’s about strategy, balance, and impact. This visual captures the key phases: ✔ From foundational steps and leadership alignment ✔ To building insights with analytics and ML ✔ To achieving flexibility and cultural transformation Avoid the traps of over-engineering and focus on creating a platform that delivers real business value. Curious to know—what’s the biggest challenge you face in your data platform journey? Let’s exchange ideas in the comments. #DataStrategy #DataPlatform #Analytics #DigitalTransformation #DataDriven

1

Metadata architecture is key to successful data catalog implementations. Don’t think of architecture as just documentation. Good metadata architecture defines all the components needed to achieve a goal and how they connect, both internally and externally. For a data catalog, that means understanding: • What metadata exists and where • How it flows across systems • How teams and other software interact with it (the most important) Architecture also communicates structure and expected behavior to all stakeholders, ensuring that everyone (business, IT, compliance, analytics, etc.) shares the same understanding. Without good metadata architecture, a catalog is just a tool. With it, it has the potential to become a connected, operating ecosystem. Note: This post was influenced by my recent visit to the Frank Lloyd Wright house with Louis Tran.

33

1 Comment

As the successful Teradata → Databricks migration project is coming to an end, I’ve decided to take a new direction within the data migration space. Over the last months, I had the opportunity to work closely with Jiri Koutny on the BW Migrator accelerator (https://migrate-bw.com/), exploring how organizations can modernize SAP BW 7.5 landscapes and transition them to modern data platforms. What makes this particularly exciting is the growing level of automation in migrations using AI, helping accelerate the transition from BW to platforms such as SAP Business Data Cloud (BDC) or open ecosystems like Databricks, Microsoft Fabric, Snowflake, and others. As part of this shift, I’ve also been deepening my understanding of the SAP data ecosystem, particularly around BDC and Datasphere, through recent SAP learning programs. The pace at which the data platform landscape is evolving is incredible, and We are working on solutions that help organizations modernize their data architecture and unlock value from their data using the latest AI-driven approaches. Looking forward to the next challenges in data platform modernization and migration. We are DataSentics, a Bull company

48