Ben Holcombe

When data agents fail, they often fail silently - giving confident-sounding answers that are wrong, and it can be hard to figure out what caused the failure. "Building and Evaluating Data Agents" is a new short course created with Snowflake and taught by Anupam Datta and Josh Reini. This course teaches you to build data agents with comprehensive evaluation built in. Skills you'll gain: - Build reliable LLM data agents using the Goal-Plan-Action framework and runtime evaluations that catch failures mid-execution - Use OpenTelemetry tracing and evaluation infrastructure to diagnose exactly where agents fail and systematically improve performance - Orchestrate multi-step workflows across web search, SQL, and document retrieval in LangGraph-based agents The result: visibility into every step of your agent's reasoning, so if something breaks, you have a systematic approach to fix it. Sign up to get started: https://lnkd.in/gVwKuCCQ

6,625

224 Comments

Demo: how to surface and use Unity Catalog Metric Views in Databricks AI/BI — to enable self-serve access to trusted business metrics. See how to set up central governance with row-level security and column masking, and more.

403

8 Comments

Over 100 enterprises are already seeing real impact from the Databricks and Palantir partnership. With Palantir Technologies' world-class AI Operating System + the Databricks Platform and Agent Bricks, customers are moving from pilots to production-scale AI deployments faster than ever. Our partnership provides a secure, scalable architecture that enables teams to deploy joint AI workflows quickly and with confidence. See what else our customers are saying: https://lnkd.in/gjFEq2HW

1,635

31 Comments

Most BI teams know they need to modernize, but few realize the cost of waiting. When we assessed our own BI environment at Databricks, we faced the same challenges many enterprises do: slow dashboards, high licensing costs, inconsistent metrics, and limited AI assistants. So we put our own platform to the test. In just five months, we migrated 1,300+ dashboards to Databricks AI/BI — improving performance 5x, saving $880K annually, and boosting user satisfaction by 80%. Here is the 5-pillar migration playbook that made it possible: https://lnkd.in/gFK6rHXT

412

11 Comments

Long Live Spark, Down with Spark 🙃 Spark has been the de facto processing engine for the last 10 years in the "Big Data" community. It still has its place in data processing. Some have branched off their own versions, spun off custom engines, and developed new approaches in an attempt to overcome the shortcomings of spark in diverse landscapes. Our focus is to enable the right compute for the problem with a focus on interoperability with other systems and to have open source compute engines as first-class citizens in platform. Enterprises are highly complex, with high heterogeneity of solutions, tech debt, data gravity, and modalities. I would argue that most pipelines in enterprise are not web scale; they are small and medium sized. A pipeline with a few million or hundreds of millions of rows, sometimes a few billion. Spark is overkill for 90%+ of these workloads and carries a lot of overhead that impacts speed and cost. With the Palantir Multi-Modal Data Plane (MMDP), we provide the framework to store data anywhere, compute data anywhere, and use any model. If we break down this a bit more, this means I can have data spread across different platforms, file formats, structures and use any compute (inside or outside of the platform) to process my data. The permutations of storage locations, formats, and compute engines are massive. It's the right tool for the problem at hand, not one tool you need to shove everything into. Re-platforming should never be the goal. With new frameworks like Polars, DataFusion, DuckDB and the availability of larger node sizes that this means you can run increasingly bigger and more complex workloads in a fraction of the time and cost. You can mix and match these as needed in Foundry and AIP, even having one step run one engine and another step use a different engine. (You can even bring your own engine.) Spark is always there on the table, but it should no longer be the default for most enterprises to start with.

232

24 Comments

Palantir & Databricks: powering end-to-end data, analytics, and operational AI. Chad Wahlquist and Benjamin Abood discuss how the Data Intelligence Platform, Unity Catalog, Foundry, and AIP all seamlessly integrate to deliver next-generation outcomes. Full interview here: https://lnkd.in/eprQUapp

665

21 Comments

If you're looking for a code-first method of deploying PBIP files or semantic models, check out this article I put together on how to get started with fabric-cicd! fabric-cicd is an amazing Python library developed by my teammates that provides a code-first method to deploy Fabric items from source control to Fabric workspaces. https://lnkd.in/evUibVGT

328

15 Comments

As we have been working with customers who are adopting Snowflake Semantic Views, one of the top requests has been a native integration with dbt Labs for materialization and referencing of Semantic Views within dbt models. I’m excited to announce that there is now a new dbt package, available on Package Hub, that does just this! You can read Yutsing Liu's blog here to learn more: https://lnkd.in/deXJtJBQ In summary, the dbt_semantic_view package introduces a new semantic_view materialization for dbt Core projects targeting Snowflake, enabling you to define business-centric semantic views (tables + relationships + dimensions + metrics) via dbt and automate their creation, renaming, and dropping. Link to package is in the comments.

365

17 Comments

Snowpipe's new pricing is rolling out to all customers now, delivering up to 50 percent savings on data ingestion automatically. 💪🏻 That spirit of efficiency defines the Snowflake AI Data Cloud. Snowpark Connect enables customers to execute Spark code natively to realize eight hundred thousand dollars in annual TCO savings. Plus, achieve 65 percent cost savings on pipelines using automated Dynamic Tables. Learn more 👉🏻 https://bit.ly/4pU9uMx

287

4 Comments

As Semantic Views become the governed layer powering BI and AI in Snowflake, teams are asking how to version, review, and deploy changes safely at scale. The answer: manage Semantic Views with Terraform and automate deployment through CI/CD. In this hands-on guide, you’ll: - Define warehouses, tables, and Semantic Views in Terraform - Enable preview features for semantic resources - Store everything in Git for version control - Automate plan and apply through GitHub Actions - Manage state centrally in Terraform Cloud Learn more: https://lnkd.in/gBDEPANT

257

16 Comments

Databricks set a new direction at the 2025 Data + AI Summit. The new mission is clear: build and deploy full-scale data and AI products on a unified, AI-native platform. Here’s what’s new: • Lakebase – Real-time workloads, fully integrated • Agent Bricks – Build, deploy, and manage AI agents • Lakeflow + Declarative Pipelines – From pipeline code to pipeline intent • Lakebridge – AI-powered onboarding, accelerated • Unity Catalog Metrics – Governed business metrics, centralized • AI/BI – Business intelligence with an AI-native interface • Databricks Apps – Serverless, data-native applications • Databricks One – Unified view of metrics, apps, and agents • Free Edition – No-cost access for small teams and prototyping We broke down what each launch means, and how they’re changing the way teams execute AI at scale. Get the details: https://hubs.li/Q03sMykq0 #DataAISummit #Databricks #Indicium #AIExecution #IndiMesh #ModernDataStack

44

Snowflake and Anthropic have expanded their partnership with a multi-year, $200 million agreement to bring Claude models deeper into the Snowflake AI Data Cloud. The deal extends access to 12,600+ customers across all major clouds and launches a global go-to-market push focused on deploying enterprise-grade AI agents. Claude will also power Snowflake Intelligence, enabling advanced reasoning and secure, governed data workflows for industries from finance to healthcare. Read more: https://lnkd.in/dVbb_AtT Follow AI Story on Instagram for the latest news and trends in AI and deeptech: https://bit.ly/46JheLd #Snowflake #Anthropic #ClaudeAI #EnterpriseAI #AIAgents

233

2 Comments

Struggling with slow queries on massive datasets? Traditional data lakes often fall short on performance, making real-time analytics a challenge. The Databricks Lakehouse combines the best of data lakes and warehouses to power faster, smarter decision-making.  ⏬ Download the whitepaper to learn how to fast-track your modernization to Databricks Lakehouse with LeapLogic: https://lnkd.in/ddBSRhAB #databricks #dataestate #modernization #cloudmigration #datamodernization #whitepaper

58

1 Comment

Snowflake: pgvector, Cortex Embed, Vector Type, and Search Service After reading Umesh Patel’s blog on Snowflake-PostgreSQL on Medium, and testing the concepts myself, I created this PDF to explain when to use various Snowflake features for building RAG applications. comments! 🤜 🤛

70

2 Comments

It would've taken 25 years for this Snowflake customer to get 100% business data quality coverage. With Anomalo, Discover now configures unsupervised data quality monitoring within 10-15 minutes for each dataset. Learn how Discover’s data team flipped the script using Anomalo on Snowflake to scale trust across petabyte-scale data: https://lnkd.in/gPjxDCFH

11