Ciju Nair, Ph.D.

It is hard to understand Fabric's long-term strategy. On one hand, it was smart to build it around PBI, the leading BI platform. On the other hand, the key to Power BI's success in dethroning Tableau was its' extremely affordable licensing. Competitive licensing is not an element that Fabric's always-on pricing model can boast of at current rates, especially when the likes of Databricks, Snowflake, and other rising platforms are around. Perhaps Microsoft will be content with Fabric always being the 3rd/4th best option?

39

32 Comments

Agent Learning from Human Feedback (ALHF) in Action A Databricks Case Study Databricks is pushing the frontier of Agentic AI by enabling its Knowledge Assistant to learn and improve through human feedback loops. By leveraging ALHF, Databricks created an agent that continuously learns from real user interactions refining responses, grounding answers in enterprise context, and evolving autonomously over time. Key Takeaways: Human-in-the-loop reinforcement makes agents more reliable and adaptive. Grounded learning boosts accuracy and trust in enterprise AI use cases. Feedback-based training ensures alignment with user intent and enterprise knowledge. A strong reminder: The future of Autonomous Agents isn’t just about LLMs it’s about learning responsibly, evolving contextually, and scaling enterprise value. #AgenticAI #ALHF #HumanFeedback #Databricks #EnterpriseAI #LLMOps #AutonomousAgents #AIinEnterprise #GenAI #AIProductDesign

13

From Strategy to Activation: Getting Your Data AI-Ready Continuing the conversation from the Snowflake World Tour, let's build off the foundational strategy with data activation. Most companies want to activate AI, but their data just isn’t ready for it. Dirty, decentralized, and inoperable data stops innovation before it starts. If your AI strategy is set, it’s time to turn to your data strategy. Here’s how to get your data ready to fuel innovation and growth: Clean it - Audit what data exists and what’s missing. - Eliminate duplicates and outdated records. - Set up continuous hygiene processes so bad data doesn’t creep back in. Centralize it - Consolidate your data into a unified, governed environment. - Define clear ownership, permissions, and metadata standards. - Eliminate silos so teams can operate from a single source of truth. Activate it - Connect clean, centralized data to the AI workflows that drive value. - Use it to train models, personalize experiences, and surface insights faster. - Track outcomes, refine what works and retire what doesn’t. You can’t build an intelligent organization on an unintelligent data foundation. AI readiness isn’t about more data, it’s about better data. If AI is the engine, is your data clean enough to fuel it? #DataStrategy #AI #Snowflake #DigitalTransformation #DataGovernance #Analytics #EmergingTech

9

2 Comments

Semantic Layer vs. MDM — Don’t Confuse the Map with the Destination. If a semantic layer gives you a “single source of truth,” does that mean you can skip MDM? Short answer: No. Longer answer: Absolutely not. The confusion is understandable - both MDM and semantic layers connect data. But they do so in fundamentally different ways, for fundamentally different purposes. As my colleague Juan Sequeda brilliantly put it: “MDM is about linking data, semantic layers are about linking schemas.” That sounds subtle. It’s not. 𝐖𝐡𝐚𝐭 𝐌𝐃𝐌 𝐃𝐨𝐞𝐬 𝐓𝐡𝐚𝐭 𝐒𝐞𝐦𝐚𝐧𝐭𝐢𝐜 𝐋𝐚𝐲𝐞𝐫𝐬 𝐂𝐚𝐧’𝐭 MDM creates trusted, governed, persistent master data — the kind you can confidently use across both dashboards and operational systems, like ERP or CRM applications. You need MDM when: ✔️ You want a golden record that’s actively governed and persisted over time in a data hub. This golden record can be used within a single domain or multiple domains - depending on the use case. ✔️ You want to ensure data used in business applications aligns with your governance policies. ✔️ Data stewards need to enforce real-world business and data quality rules at a record level, not just metadata mappings. ✔️ You need to transform inconsistent and sometimes inaccurate source data into shareable, policy-aligned truth. ✔️ Your “truth” varies by context - and your governance model reflects that. This isn’t just about control - it's about visibility and explainability for the data that is shared widely across your organization. 𝐖𝐡𝐚𝐭 𝐒𝐞𝐦𝐚𝐧𝐭𝐢𝐜 𝐋𝐚𝐲𝐞𝐫𝐬 𝐃𝐨 𝐑𝐞𝐚𝐥𝐥𝐲 𝐖𝐞𝐥𝐥 Semantic layers are powerful analytical accelerators. They abstract complexity, unify definitions, and streamline data access - but they don’t manage data. You may need a semantic layer when: ✔️ You need fast, federated query access across diverse sources. ✔️ You don’t need to change or persist the data. ✔️ You just want to standardize definitions — not enforce governance or correct quality issues. ✔️Your use case is read-only — think dashboards, not transactions. In short: Semantic layers help you to have a consistent view of the data, and MDM helps you actively govern and manage it. 𝐖𝐡𝐲 𝐓𝐡𝐢𝐬 𝐌𝐚𝐭𝐭𝐞𝐫𝐬 𝐍𝐨𝐰 Semantic layers are having a renaissance, thanks to all the talk of data fabrics, data products, and AI-ready architectures. They play a critical role in helping to create a 'context layer' that will be critical to enable AI to leverage insights buried in highly structured data. But let’s be clear: A semantic layer without MDM is like a beautifully labeled map… but where the cities on the map are duplicated or mislabeled. This means that any next generation 'context layer' must necessarily include *𝐛𝐨𝐭𝐡 𝐚 𝐬𝐞𝐦𝐚𝐧𝐭𝐢𝐜 𝐥𝐚𝐲𝐞𝐫, 𝐚𝐧𝐝 𝐌𝐃𝐌*. What do you think? Piethein Strengholt Ole Olesen-Bagneux Jean-Georges Perrin Jessica Talisman Jon Cooke Ryan Dolley Samir Sharma Joe Reis Helen Grimster

666

153 Comments

Everyone says they have "Single Source of Truth." But your Imaging data is in S3 buckets. Commercial in Snowflake. Genomic arrays across 6 other systems. You don’t win with more data, you win with usable data. Everyone says they are “data-driven.” Here’s the reality: 12 systems. 6 clouds. Zero trust in data Not an AI failure, an architecture failure Walk into any mid-sized biotech or pharma right now and you’ll see the same thing: R&D runs on legacy genomics platforms Clinical data lives in proprietary systems. Imaging data sits in S3 folders with no metadata. Commercial analytics lives in Snowflake or Databricks Everything’s in silos but nothing talks. So when your head of research says, “Let’s run a cross-study model to predict responders,” The data team goes silent They know it’ll take three months just to merge datasets. Half that time will be spent just defining what “sample” even means across systems. That’s duct-tape engineering, it kills velocity Here’s the trap everyone falls into: You “modernize” by adding another tool. Each tool solves a local problem (ETL, storage, analytics). Integration complexity doubles. Everyone blames the data team, but it’s data architecture debt. The next generation of life sciences, pharma companies aren’t chasing “modern stacks.” They’re building universal data architectures, one foundation that handles every data type, every team, and every workflow. That means: Tables + images + genomics + vectors → one data structure. Governance + lineage + access → one policy layer. Analysis + AI → one compute engine. Designed to work together from day one, that’s the universal model Here’s how top biotech and pharma teams are pulling this off: 1. Map the chaos. Inventory every dataset, every format, every owner. Group them: Tabular Scientific arrays Imaging Text AI/Vector data Quantify what’s actually being used vs. what’s just rotting in cloud storage. 2. Standardize definitions Most treat governance as a separate initiative. But it makes all work “Sample,” “gene", all defined once, globally. Centralized authentication across all datasets. Version control, enabled by default Unified access policies that span data types. When an auditor asks, “Where did this result come from?” you can answer it instantly. 3. Universal infrastructure Move from “stacks per department” to one universal foundation that can store and query any data type. With multidimensional arrays - a data structure that can represent any modality can be stored and processed This eliminates the need to constantly convert between formats. 4. Align to ROI Once your data lives in a unified structure, you can query across modalities in seconds For example: Combine genomic data (arrays) with clinical trial data (tables) to identify responders Or imaging + transcriptomics to accelerate biomarker discovery If dealing with multi-modal complex data, repost and check our newsletter on it

58

15 Comments

Ready to build AI agents? We have launched two new courses — AI Agents Fundamentals and Get Started with AI Agents — to help you understand how agents work, and unlock value for your organization. Earn a Databricks badge upon successful completion to validate your AI skills. Whether you’re exploring AI agents for the first time or want to start building today, these courses give you the foundation to move from theory to deploying your first agent.

27

1 Comment

I struggle to write this post because the content is self-referential. It’s genuinely difficult to succinctly capture the potential of agentic workflows. But here is the breakthrough: Today I integrated Mixpanel’s MCP into Claude Code. On its own? A nice utility. Combined with my other agents? A closed-loop product team. My current stack: • Design System Agent: Reads/writes to Figma. • Design Reviewer Agent: Audits the output. • Product Manager Agent: Writes user stories and understands briefs. With the Mixpanel integration, I can now build a Measurement Agent that quantitatively tracks what we release. The Loop: The Measurement Agent feeds data to the PM Agent → The PM identifies friction → The PM writes targeted design tickets → The Design Agent executes. The hardest part isn’t the tech, rather, it’s not trying to do everything at once. The "surface area" of what's possible grows every single day. What a time to be alive. #AgenticAI #BuildInPublic #ProductManagement #MCP #FutureOfWork

9

4 Comments

It would've taken 25 years for this Snowflake customer to get 100% business data quality coverage. With Anomalo, Discover now configures unsupervised data quality monitoring within 10-15 minutes for each dataset. Learn how Discover’s data team flipped the script using Anomalo on Snowflake to scale trust across petabyte-scale data: https://lnkd.in/gPjxDCFH

11

Two new #SAS macros have been added to #SASjs core: * mf_mimetype - returns the mime type for a given file extension * mfv_getpathuri - returns the URI of a #SASVIYA file or folder for a given path We also refactored mv_createfile (creates a file in SAS Drive) to use PROC HTTP instead of FILENAME xxx FILESRVC to fix a couple of things: - enable reading / writing from one SAS Drive file to another - enable the creation of streaming HTML files using SAS code The docs have been updated at https://core.sasjs.io and the #SASPACKAGE has been updated thanks to Bartosz Jabłoński !

40

1 Comment

🚀 BigQuery Just Got a Major Upgrade! If you're working with large datasets, advanced analytics, or vector search, this is big news. 🧠📊 📌 New Features in Preview: ✅ VECTOR_INDEX.STATISTICS – Detect data drift in indexed tables ✅ ALTER VECTOR INDEX REBUILD – Rebuild vector indexes when needed ✅ Access Transparency – Now supports data prep in GA ✅ New LOAD DATA Options: • null_markers for better NULL handling • source_column_match for smarter column mapping ✅ MATCH_RECOGNIZE Clause – Powerful pattern recognition directly in SQL! 💡 These features make BigQuery even more powerful for ML, analytics, and clean data ingestion workflows. Whether you're building AI search, optimizing ETL, or analyzing behavior patterns these tools will level up your data strategy. #BigQuery #GoogleCloud #DataAnalytics #VectorSearch #SQL #MachineLearning #DataEngineering #JobairMahmud #ServerSideTracking #DataDrivenMarketing #CloudAnalytics #DataDrift #GCP #GA4

26

1 Comment

tldr; I share some thoughts on what to do when you notice errors in a benchmark dataset you are using for evaluating the capabilities of large language models in some context. #llmevaluation #benchmarking #dataquality https://lnkd.in/gDbcsEjm

25

For a long time, “self service” in Tableau meant something very specific. Business users became tool users. They learned how to drag, drop, filter, join, model. Self service was really about learning the interface to get to the data. That model is starting to break. With agentic AI, we are moving toward a very different form of self service. One where business users no longer need to know how the tool works. They only need to express intent. “Just ask what you need.” I did a small experiment using the Claude browser plugin. It can observe and operate your browser, which is both impressive and something to be careful with. The setup was simple. A business user creates a rough mockup of a visualization or dashboard. Just an image. That image is uploaded into Tableau, connected to a dataset that could realistically support the request. What happens next is the interesting part. Modern AI models can already interpret what they see in that image. They understand layout, intent, chart types, measures versus dimensions. From there, they can start operating the interface themselves and build toward the desired result. It is not ultra fast yet.But the direction is very clear. I am not saying Claude is the next logical add-on to your Tableau analyst stack. That is not the point. With upcoming Tableau APIs that allow programmatic control of the platform, this feels like a sneak preview of where analytics tools are heading. Less focus on teaching people tools. More focus on letting people express intent and letting systems do the rest. Self service is evolving. And it no longer looks like drag and drop. https://lnkd.in/eDCM6Aks

32

4 Comments