Navnit Shukla

A great video explaining what RTM is, what architectural changes enable the ultra-low latency processing, some customer references and a demo! 🤓

15

Exciting news! Dun & Bradstreet is powering agentic AI workflows built by Agent Bricks through Model Context Protocol (MCP) servers on Databricks Marketplace. Anchored by the D-U-N-S® Number, this integration reduces hallucinations, enhances decision accuracy and accelerates the deployment of agentic systems. More to come ✨

4

Databricks launches 'Genie Code,' an innovative AI system that autonomously manages and operates enterprise data pipelines and analytics. Unlike conventional AI coding tools focused on software writing, Genie Code understands data structures, automates workflows, and integrates governance policies, enhancing reliability and efficiency. Early adopters report significant productivity improvements, as this technology reshapes the future of data engineering by augmenting human expertise rather than replacing it.

1

1 Comment

The initiative introduces a vendor-neutral semantic model specification designed to create consistency in how business logic is defined and shared across AI and business intelligence applications. #dbtlabs #snowflake #blackrock #salesforce https://lnkd.in/dKs8h8Tg

15

Ready to unlock the power of AI in Snowflake? In this episode of Behind the Cape, Data Superheroes Keith Belanger and Karthik Raman take you on a deep dive into Cortex AI—covering Snowflake Copilot, Document AI, AML Studio, and more. 🎧 Tune in to hear how Cortex can streamline your AI + data workflows: https://lnkd.in/g5EfHQkv 🔍 Explore more: developers.snowflake.com

133

1 Comment

what if you could run AI on your Databricks data: without moving a single row? this pattern in the Interoperability Series covers how CLD + Cortex AI is simple! No GPU clusters | No model endpoints | No MLOps pipeline. Just SELECT! P1 (CLD) + Cortex AI + P2 (SF-Managed Iceberg write-back) = the full AI interop loop. ➡️ https://lnkd.in/g6c4xnDc #CortexAI #ApacheIceberg #Interoperability Snowflake

14

🚀 Snowflake Dual-Mode Document Parsing with Cortex AI Many document pipelines fail because they must choose between text accuracy and layout understanding. This hybrid approach avoids that trade-off by running OCR Mode and Layout Mode in parallel, then merging results using Claude Sonnet. How it works • Layout Mode captures tables, sections, hierarchy • OCR Mode improves raw text accuracy • Claude Sonnet performs AI-assisted merge • Snowflake Cortex COMPLETE function extracts business attributes for analytics Pros ✅ Higher field-level accuracy ✅ Clean, structured, analytics-ready output inside Snowflake ⚠️ Slightly higher compute cost ⚠️ More orchestration than single-mode parsing #snowflake #snowflakesquad #snowflakecortex, #snowflakeEDAI #Genpact Here is the high level documentation:

4

Modern data stacks rarely live in one system. That’s why we've expanded Cortex Code CLI beyond Snowflake workflows to support any data, anywhere, starting with dbt and Apache Airflow® patterns natively. From schema-aware discovery to DAG generation and debugging, it brings deeper context to multi-system data engineering – all while maintaining enterprise governance and model flexibility. And, we're opening access with a self-serve subscription model, so you can get started without a Snowflake account. Details here: https://lnkd.in/gNf7gjz5 

11

Switching between tools is still a big part of data workflows. This integration brings Databricks into Cortex Code, making it possible to explore, build, and debug without leaving the same environment. More from Dash DesAI: https://lnkd.in/gR_6Cmue

9

Snowflake is releasing so many AI features lately, it's really hard to keep track of them all. For AI Agents, evaluation is very important to improve their performance. But setting up an evaluation system yourself takes a lot of work. Fortunately, the Evaluation feature for Snowflake's Cortex Agent is now in GA! If users just need to prepare evaluation data they can utilize this built-in feature to run continuous evaluations. In my blog post, I explain how this feature basically works, and also share my favorite part: how to run it and get the evaluation results using just SQL. [medium]: https://lnkd.in/gyj-Qu9n [zenn]: https://lnkd.in/gzMV2k_P

52

Pretty excited about this release.... AWS just enabled a powerful new way to unify data governance across Amazon Web Services (AWS) and Snowflake with catalog federation between @AWS Glue Data Catalog and Snowflake Horizon Catalog. This makes it possible to discover and query Iceberg tables in Snowflake Horizon directly from AWS analytics services, with fine-grained access controls handled by AWS Lake Formation. https://lnkd.in/gNTtsm5d #Data #AWS #Catalog #Snowflake

45

Great news for Spanner and BigQuery users. Datastream now provides a seamless, serverless bridge from Spanner’s transactional power to the high-performance analytical capabilities of BigQuery and BigLake. Get the most scalable and available database that runs Google with BigQuery and BigLake’s serverless AI-Native Lakehouse!

517

3 Comments

Watch this video to learn how my team manages non human identities and how Aembit plays a great part in the workload to workload connectivity.

10

Claude, ChatGPT, Snowflake Intelligence, Cortex Analyst, Databricks Genie, Zenlytic, TextQL. Think about any AI agent you connect to your data. They all get questions from business users like: “What was the average number of daily active users in the last quarter?” These AI agent connects to Euno’s MCP server to retrieve metadata. That’s when Euno gives these agents the missing context they need to understand your data. ** Just a reminder that Euno is a metadata platform so we only access your metadata ** Euno runs a semantic search and looks for dashboards that use DAU (which is semantically equivalent to Daily Active Users). It filters for certified dashboards using the trust signals you’ve already set in Euno. In this example Euno finds two candidates: 1. DAU used in the User Engagement dashboards 2. DAU_NEW used in the Product Statistics dashboard Both dashboards are certified. But Euno also shows how often each has been used in the past 30 days, along with the owner or creator. With that information, the business user can confidently choose the right one and the AI agent can deliver the correct answer. The same applies to tools like Cursor. Let’s say you’re an engineer and ask: “Can you find me the Tableau dashboard for Customer Success?” Cursor will reach out to Euno’s AI assistant to get that BI context because Cursor is focused on dbt development and doesn’t have access to that layer. You can even go further and ask which dbt models power that dashboard and how those models are calculated. Euno feeds Cursor with that real-time context as you build. It’s not just for business users. It’s for developers too.

25

5 Comments

Great news, Snowflake Cortex Agents API is now GA! Developers can now build trusted, production-ready data agents that orchestrate insights across structured and unstructured data using Cortex Search and Cortex Analyst. Build context-rich agents today: https://lnkd.in/d7GNsEwj

3

🚀 Rethinking Data Backups in Databricks: Why Declarative LakeFlow Pipelines Are Becoming the New Default As data engineers, many of us start backups the “classic” way: Spark Structured Streaming readStream + writeStream manual checkpointing, custom triggers, and a lot of glue code. “It works, but the operational overhead grows quickly, especially when you need long‑term data retention or continuous history for your tables.” Over the last few months, I’ve been exploring simpler and more scalable approaches for backing up data in Databricks. During this process, I revisited a pattern many of us rely on: 👉 Spark Structured Streaming with manual checkpointing It’s powerful — but not always the simplest or most maintainable. While researching, I discovered something interesting-> ➡️ Spark Declarative Lakeflow + Streaming Tables, there’s now a much cleaner option for a lot of simple append‑only backup scenarios – not just for system tables, but for any tables where you want to maintain continuous history. Key benefits I found: 💡 Automatic checkpointing & state management No need to maintain checkpoint paths, failure recovery logic, or job orchestration manually. 💡 Cleaner scheduling Simply declare the refresh interval — no clusters, triggers, or job wiring required. 💡 Stronger governance Everything is tracked in Unity Catalog with built‑in lineage. 💡 Less code, less failure surface Perfect for straightforward, long‑term replication. And yes — Structured Streaming is still useful for advanced scenarios (custom triggers, multi-sink writes, foreachBatch, etc.) that are not yet supported in Spark Declarative LakeFlow. But for most backup pipelines, LakeFlow Streaming Tables offer a simpler approach that “just works.” Where this approach fits best ✔️ Append‑only datasets ✔️ Tables where retention limits are too short ✔️ Teams who want durable, low‑ops backup pipelines ✔️ Any data source where simplicity > tuning micro‑batch internals If you're still using manual Spark Structured Streaming for simple ingestion/backups, it’s worth looking at Streaming Tables in LakeFlow. The operational simplicity is a game changer. Happy to share more details or code patterns if anyone is exploring similar use cases. #Databricks #DeltaLake #Lakehouse #DataEngineering #Spark #LakeFlow #StreamingTables

16

We tend to run ML pipelines sequentially by default. This example from Dash DesAI shows how agent teams can handle the same workflow in parallel with less orchestration, more autonomy and faster results. Great perspective 👇

1