Robin Moffatt

Claude Sonnet 4.5 is now available on Databricks! Anthropic's most powerful model yet is built for real-world agents, handling complex, long-horizon tasks across coding, finance, research, and cybersecurity. And now it’s available to be applied directly to your governed enterprise data in the Databricks Platform. With Claude 4.5 on Databricks, customers can: • Build domain-specific agents with Agent Bricks • Run large-scale analysis directly from SQL and Python with AI Functions • Automate GenAI ETL pipelines in Lakeflow With Databricks, you get the flexibility to run the best models securely on your data and scale as needs evolve https://lnkd.in/gs3GpaXg

2,593

51 Comments

At #SnowflakeBUILD London, we just set a new bar for the developer experience. We’ve rolled out a data-native ecosystem that collapses the distance between your first line of code and your first production deployment. What's new: - Cortex Code: Accelerate everything from complex data engineering to agent development with built-in AI assistance. - Shared Workspaces: Your new home base. Bring code, notebooks, and dbt projects into one collaborative view. - Snowflake Notebooks v2: The power of Jupyter meets the scale of Snowflake, now with enhanced ML performance. This isn't just a feature drop. It’s about removing the friction that kills innovation. Build, test, and ship, all without leaving the AI Data Cloud. See the full developer toolkit: https://lnkd.in/gpYbf4CC

190

Delivering AI-ready data is harder than ever — too many tools, manual code, and pipelines that don’t scale. Join Databricks’ Bilal Aslam and Simon Whiteley, CTO at Advancing Analytics, to see how a unified, declarative platform helps you: - Replace repetitive code with tested pipelines - Support batch, streaming, and unstructured data - Monitor pipeline health with built-in quality checks, schema evolution, and error handling You’ll leave with a clear plan to simplify your stack, scale your workflows, and deliver reliable data for AI: https://lnkd.in/g2U4gW9h

205

5 Comments

We’re excited to announce full Apache Iceberg™ support in Databricks, unlocking the full Iceberg and Delta Lake ecosystems with Unity Catalog! This is a major advancement toward a single, unified open table format. Now you can: - Use Databricks and external engines to read and write Iceberg tables managed by Unity Catalog - Access and govern Iceberg tables managed by external catalogs Get the details here: https://lnkd.in/gMQ2Gd9s Tune-in to #DataAISummit to hear Matei Zaharia share more.

1,005

13 Comments

What I love about #ApacheIceberg is how many different workloads folks are running on the same copy of data. Graph queries are a really cool example. Instead of copying data into specialized graph databases, PuppyGraph lets you run graph traversals directly on your existing Iceberg tables. Pretty cool stuff. But here's the thing - to get truly great performance, you need to ensure your Iceberg tables are well maintained and optimized. That's where Ryft comes in. We make sure your Iceberg tables are automatically maintained and optimized, no matter what engine you are using. It was terrific to work with Zhenni Wu and Weimo Liu, to create this blog post on how you can use Ryft with PuppyGraph to run graph queries on a fully optimized Iceberg lake! https://lnkd.in/giwnwMuc #ApacheIceberg #GraphQueries

100

2 Comments

Snyk used to spend hours searching for internal info. Now, their team gets instant answers to complex queries using a Cortex AI chatbot—in pure SQL. ⚡ By centralizing data on Snowflake and building a simple chatbot, Snyk saves 1,250 hours a month and empowers non-technical teams to get the insights they need. This is what it looks like when enterprise AI gets real: actionable, scalable, and built where your data already lives. Check out the full story: 👉 https://bit.ly/47lQfWh

221

4 Comments

Databricks Performance Optimization: Speed That Saves Money In Databricks, performance isn’t just about faster queries — it’s about using compute wisely. When pipelines aren’t optimized, costs rise quietly in the background. So what actually makes a difference? 🔹 Smarter data layout Well-organized data means systems scan less and respond faster. 🔹 Balanced data distribution Too many splits slow things down. Too few limit growth. Getting the balance right improves both speed and scalability. 🔹 Reuse instead of reprocessing Data that’s accessed often should be reused, not recalculated every time — saving time and cost. 🔹 Right-sized compute Bigger infrastructure doesn’t always mean better performance. The right size delivers the same results at a lower cost. 📊 The business impact Faster data pipelines Lower cloud spend More stable performance as data grows Well-optimized platforms don’t just run better — they pay for themselves. 🚀 Fast data isn’t expensive data — when it’s engineered the right way. #Databricks #DataEngineering #CostOptimization #ScalableData #Lakehouse #CloudEfficiency

21

Tuesday's webinar with Orchestra on Apache Iceberg was pretty cool and a huge thanks to everyone who joined! 🙌 The questions we received really showed the appetite for practical, vendor-agnostic data infrastructure. Here are my 3 fav ones: - "What warehouse capabilities does Iceberg bring to object storage?" The big ones: ACID transactions for consistent writes, schema enforcement and evolution, time travel for debugging and compliance, and optimized queries with partition pruning and predicate pushdown—all without vendor lock-in. - "Why is Iceberg becoming so important for data engineers?" We're seeing a massive shift from warehouse-first to lakehouse designs. Iceberg is becoming the lingua franca for modern data stacks—Snowflake alone reported a 10X increase in companies using Iceberg in less than a year. - "Does time travel work on top of data branches?" Time travel is built into bauplan's data branches. Similar to Git, a branch is just a head of a commit chain with a name. While the branch is mutable, the commit history isn't. You can time travel to any past version or run pipelines against historical states. Missed the webinar? Here is the recording https://lnkd.in/gAWTWG28

32

2 Comments

Great news for Spanner and BigQuery users. Datastream now provides a seamless, serverless bridge from Spanner’s transactional power to the high-performance analytical capabilities of BigQuery and BigLake. Get the most scalable and available database that runs Google with BigQuery and BigLake’s serverless AI-Native Lakehouse!

504

3 Comments

🎥 Shubham Mali breaks down how Pi-Ingest is transforming data engineering in the Snowflake era! In this video, Shubham shares how one of our enterprise clients cut data ingestion costs by over 60% by ditching legacy ETL tools and adopting Pi-Ingest — our fully Snowflake-native ELT framework. 🔹 No third-party licensing 🔹 No extra compute costs 🔹 100% automation — from ingestion to transformation 🔹 Config-driven. No code. Just scale. From SCD1/2/3 to fact/dimension tables and auto-scaling, Pi-Ingest simplifies your architecture while accelerating time to insights. ⚡ Fast. Flexible. Built for Snowflake. #PiIngest #Snowflake #DataEngineering #ELT #CloudData #ModernDataStack #PibyThree #ShubhamMali #NoCode #CostOptimization

85

5 Comments

🚀 2026 is shaping up to be a pivotal year for Databricks innovation. In this short video, I walk through three products every Databricks partner should have on their radar — and a plan for — to stay ahead of the curve: - Agent Bricks – automate the grind of building, evaluating and optimising production-grade AI agents on your own data.​ - Databricks Apps – securely build and run data & AI apps directly on Databricks with full governance and zero infra hassle.​​ - Lakebase – a fully managed Postgres OLTP engine, tightly integrated with the lakehouse, built for AI-native applications and agents These are the high‑growth areas where we’re seeing transformative potential across industries. Stay tuned, stay curious — and make sure your 2026 strategy keeps pace with the innovation curve. #Databricks #AI #Lakehouse #Agentbricks #DatabricksApps #Lakebase #DataAndAI #TechTrends2026 #MachineLearning #DataEngineering #AITransformation

71

6 Comments

Quartr API datasets are now available via Snowflake. Access structured earnings data – live audio, real-time and historical transcripts, filings, slide decks, and event summaries. Pull Quartr data directly into your existing Snowflake workflows and augment your datasets. No new pipelines, no separate API integration. Build with data that drives decisions.

75