John Kang

Most BI teams know they need to modernize, but few realize the cost of waiting. When we assessed our own BI environment at Databricks, we faced the same challenges many enterprises do: slow dashboards, high licensing costs, inconsistent metrics, and limited AI assistants. So we put our own platform to the test. In just five months, we migrated 1,300+ dashboards to Databricks AI/BI — improving performance 5x, saving $880K annually, and boosting user satisfaction by 80%. Here is the 5-pillar migration playbook that made it possible: https://lnkd.in/gFK6rHXT

413

11 Comments

🚀 Big news for Databricks Assistant users — AI just got way better at editing your code. Introducing Assistant Edit Mode 🧠💻 — a new way to apply AI-generated suggestions across multiple cells at once. Now you can: ✅ Refactor logic, rename variables, and clean up code — all with a single prompt ✅ Migrate SQL, Pandas, or Python code faster and more accurately ✅ Standardize style, remove clutter, and even generate test scaffolding automatically No more tedious, cell-by-cell edits. Edit Mode understands your entire notebook context — so your changes are consistent, fast, and easy to review. In early testing, it cut refactoring time by over 50%. This is just the start. You will soon hear about Edit Mode in AI/BI Dashboards and how we are integrating agent-like workflows into Databricks Assistant. Stay tuned! 📖 Read the full blog and see it in action 👇 #Databricks #GenerativeAI #DataScience #MachineLearning #DataEngineering #Productivity #AI

4

Analyze Slow Queries in ‪Snowflake using Revefi’s AI Agent Watch this short video to see how Revefi’s AI Agent, RADEN, identifies and analyzes slow-running queries that consume excess compute credits, delay workflows, and signal warehouse or SQL inefficiencies across your Snowflake environment. Our innovative technology empowers you to: - Detect queries with high queuing or long execution times impacting performance and cost - Understand query patterns, user activity, and warehouse configurations contributing to latency - Receive actionable recommendations such as warehouse resizing, multi-clustering, workload separation, and query scheduling to boost efficiency See our Agentic AI in Action Without Any Setup Watch how to transform your Snowflake performance management and query optimization quickly. #FinOps

10

Getting data into Snowflake is one thing.    Getting it there quickly and reliably is what makes the difference for analytics and AI.    This live session shows how teams are improving that process when working with SQL Server.    You’ll see how to:  • Reduce delays in data delivery  • Simplify how pipelines are built and managed  • Support more up-to-date reporting and analytics 

8

Federal data transformation is happening at mission speed. Last week during the Databricks Data + AI World Tour, CEO Ron Ash sat down with Rory Patterson, General Manager of Databricks Federal, for a Tech on Deck conversation that showcases exactly how we're delivering together—from breaking down data silos to achieving true interagency interoperability with Delta Sharing. This isn't about pilots or studies. It's about Monday calls becoming Friday solutions. Watch the teaser and stay tuned for the full episode in January. #Partnership #Databricks #AIFoundation #DataArchitecture #FederalReinvention #Data #AI #TechonDeck

153

5 Comments

Why compute relationships when you can store them? That’s the core of graph thinking. Michael Hunger, VP of Product Innovation at Neo4j, breaks down how their platform treats connections as data and how that changes what’s possible for supply chain, security, and more. Tune in to his full conversation with Simon Maple.

31

2 Comments

Arguably one of the biggest breakthroughs yet in democratizing access to data and insights. Genie gives every employee their own personal conversational data analyst and date scientists. The speed and depth of how work gets done has changed forever. The ability to ask complex business questions in natural language and instantly explore insights fundamentally changes how organizations operate and make decisions. The world is changing quickly, and this is a real glimpse into the future of work. Love Genie

29

Snowflake is more than a data warehouse; it’s a unified AI data platform where AI comes to your data, not the other way around. Skip the Day Zero “Frankenstein stack” and scale with secure, elastic AI built right in. 🤖📊 With capabilities like Cortex Search for unstructured data, semantic views, Snowflake Intelligence, and Cortex Code, you can unlock conversational BI, governed text-to-SQL, and agentic AI, all within a single platform. Curious how data leaders are putting this to work today? Read the full Q&A with James Newsom ✨https://lnkd.in/gdS28DS2

14

1 Comment

Databricks' Instructed Retriever significantly advances enterprise data retrieval by integrating metadata understanding, achieving a 70% improvement over traditional methods. This innovation improves AI agents' ability to process complex, instruction-based queries and manage rich metadata, setting a new standard for enterprise AI systems that handle diverse and structured data.

1

1 Comment

If your Databricks environment has grown without guardrails, now is the time to assess. XTIVIA’s Databricks Insight360 ToolKit leverages continuous, adaptive Agentic AI to move beyond static health checks with real-time monitoring, proactive recommendations, and optional self-healing. The 1 to 2 week assessment helps ensure your environment stays secure, compliant, and optimized for cost efficiency while delivering industry-specific insights for Retail, Insurance, Energy, and Banking. Learn more in our blog: https://bit.ly/4c3zuS5 #Databricks #AgenticAI #DataGovernance #AI #CloudOptimization #EnterpriseAI #DigitalTransformation

13

🔥 Snowflake Feature Spotlight: Streams + Tasks = Real-Time Data Pipelines One of the most powerful (and underrated) capabilities in Snowflake is how easily you can build near real-time pipelines — without external orchestration tools. Here’s a simple pattern I’ve been exploring: 👉 Streams → Capture CDC (INSERT / UPDATE / DELETE) 👉 Tasks → Automate processing (scheduled or triggered) 👉 Merge Logic → Maintain SCD Type 2 history Why this is powerful: ✔ No need for external tools like Airflow for many use cases ✔ Built-in change tracking (no complex CDC setup) ✔ Scalable and serverless execution ✔ Perfect for incremental data processing Real-world use case: Imagine ingesting customer updates from multiple systems — using Streams + Tasks, you can: → Capture every change → Process only new data → Maintain full historical tracking (SCD2) All inside Snowflake. This is where Snowflake really shines — simplicity + power. 💡 Curious: Are you using Streams & Tasks in production, or still relying on external schedulers? #Snowflake #DataEngineering #Streams #Tasks #SCD2 #ModernDataStack #ELT

6

Is your Databricks query running at full speed? Most teams rely on Z-ORDER and partitioning for performance, but there’s a new contender: Liquid Clustering. The difference isn’t just technical—it’s the gap between waiting minutes and getting answers in seconds. Curious how Liquid Clustering can supercharge your Azure workloads? I break down the real-world impact, when to use each, and how to avoid costly slowdowns. Ready to see the numbers? Visit gambilldataengineering.com or message me for a practical walkthrough. 🚀

1

Many data teams have invested heavily in Apache Spark—but maintaining clusters, managing jobs, and optimizing performance is a constant challenge. That’s why Snowflake created Snowpark—and why Infostrux built the Spark to Snowpark Migration Accelerator. We simplify the move with: - Automated workload conversion - Performance tuning on Snowflake’s elastic engine - A future-ready foundation for ML, batch, and streaming data It’s not just a migration—it’s a modernization. With Infostrux, you can transition Spark workloads seamlessly into the Snowflake Data Cloud—while improving speed, security, and scalability. Learn how we do it: 🔗 https://hubs.ly/Q03vHxJ70 #SparkToSnowpark #SnowflakeDataCloud #DataPlatformModernization #SnowparkMigration #InfostruxAccelerators

9

Last week we announced the launch of @Snowflake native semantic views at #snowflakesummit.  Snowflake Semantic Views make it easy to define consistent metrics and dimensional context—right inside Snowflake. Whether you're working with BI tools, AI agents, or just trying to reduce logic drift across teams, Semantic Views provide a governed, SQL-native layer that scales. Feedback from customers and partners has been very positive, and there are lots of questions on how to get started. To help you get hands-on, we’ve launched a Semantic Views Quickstart. It walks you through: 🛠️ Modeling — Defining semantic models using SQL 📊 Metrics — Creating measures with built-in dimensional awareness 🔍 Querying — Exploring semantic views directly or with Streamlit 💬 Natural Language — Asking questions with Cortex Analyst Link here: https://lnkd.in/gMYGWRHf If you're evaluating how to centralize and scale your semantic layer within your existing Snowflake environment, this is a great place to start. Try it out and let us know what you build. #Snowflake #SemanticLayer #DataModeling #BI #Analytics

116

Snowflake credits aren’t just compute. They hide in storage too. Unused dev/test databases, stale tables, and long Time Travel or Fail-Safe settings silently drain budgets every month. Many teams never notice until credit burn rates spike. A recent Snowflake case study showed that 40% of storage in a large enterprise came from inactive data objects. That’s not innovation — that’s waste. At Anavsan, we built an AI-driven way to stop it. Our platform automatically scans your Snowflake environment, identifies idle tables, clones, and retention bloat — and helps you reclaim credits within weeks, not months. -- Pinpoint inactive storage -- Automate cleanup and governance using a secure workspace -- Reuse saved credits for new workloads Read the full blog: https://lnkd.in/eXUFNgkd Because in FinOps, every reclaimed credit fuels what really matters — growth, not waste. #FinOps #Snowflake #CloudOptimization #DataEngineering #Anavsan #CloudCostOptimization

2

“It’s not whether you can build the system. It’s whether you can build the right one.” Modern tooling makes it easy to stand up production-grade data systems. The harder problem is defining shared concepts across an entire firm and ensuring those definitions hold as the organization grows. In our latest Substack post, we write about how data strategy has shifted from a technical challenge to a business one, and what that means in practice. 🔗 https://lnkd.in/ej3caCXp

10

🔥 Data isn’t just the new oil—it’s rocket fuel. And Databricks is our launchpad.🔥 From wrangling messy datasets to powering real-time insights, our team has been pushing boundaries with Databricks—and the results speak for themselves. 💡 Whether it’s building lightning-fast pipelines with Apache Spark, simplifying governance with Unity Catalog, or scaling machine learning with MLflow, we’re turning complexity into clarity. Here’s what’s exciting us right now: ⚙️ Seamless cloud integration (Azure, AWS, GCP) ⚡ Real-time data processing that actually delivers 📚 Collaborative notebooks that make teamwork feel effortless 🔐 Enterprise-grade security and governance baked in We’re not just using #Databricks—we’re building with it, experimenting with it, and evolving through it. 🚀 If you're exploring modern data platforms or just curious how Databricks can transform your data game, let’s #connect. We love sharing what we learn. #Databricks #ApacheSpark #DeltaLake #MLflow #DataEngineering #CloudComputing #BigData #AI #MachineLearning #TechInnovation #DigitalTransformation #StreamingData #DataScience #FutureOfData #asyva

6