Tim Costello

Demo: how to surface and use Unity Catalog Metric Views in Databricks AI/BI — to enable self-serve access to trusted business metrics. See how to set up central governance with row-level security and column masking, and more.

403

8 Comments

Data should drive business impact, not frustration. One of our top senior data scientists authored a simple and succinct guide to help your data projects succeed. In this guide, you’ll learn:  ✅ Why most companies find themeselves in chaotic data projects  ✅ The exact steps to optimize ROI based on data  ✅ A proven framework for making better, swifter decisions Download the guide free 👉 https://lnkd.in/entY8yam 

12

Most BI teams know they need to modernize, but few realize the cost of waiting. When we assessed our own BI environment at Databricks, we faced the same challenges many enterprises do: slow dashboards, high licensing costs, inconsistent metrics, and limited AI assistants. So we put our own platform to the test. In just five months, we migrated 1,300+ dashboards to Databricks AI/BI — improving performance 5x, saving $880K annually, and boosting user satisfaction by 80%. Here is the 5-pillar migration playbook that made it possible: https://lnkd.in/gFK6rHXT

412

11 Comments

Is your team drowning in data pipeline requests while your GenAI projects are stalled? You don't have an insights problem; you have an action problem. The bottleneck is the manual work connecting your data to your AI. In this article, we explore how Systech’s #WizarD™, operating within the #Snowflake AI Data Cloud, transforms data engineering from a manual, resource-intensive process into an autonomous, optimized, and scalable function. Read full article: https://lnkd.in/eUmzzq4n This is more than an advancement in technology; it is a fundamental redefinition of what it means to be a data-driven organization. The age of intelligent action is here. See how WizarD™ Analytics works. https://lnkd.in/efBrKTpG Aditya G., Nikhila Preethee Ravichandran , Sunil Kumar, Vaishnavi Gollapudi Venkat Raghavan R. #AgenticAI #GenAI #DataEngineering #Automation #DigitalTransformation #Systech

11

🛠️ Every DBA needs a toolkit—this one feels like having a whole team of experts on call. One of the most impressive community contributions in the SQL Server world is Brent Ozar’s First Responder Kit. 👉 What it does: sp_Blitz → instantly tells you what’s wrong with your SQL Server. sp_BlitzCache → shows why queries are slow. sp_BlitzIndex → helps you understand and fix indexing problems. sp_WhoIsActive (included) lets you see what’s happening right now. Instead of spending hours digging through logs, you can run a procedure and get a clear, prioritized report—like having a first responder on the scene of a database emergency. ⚡ I’ve used it, and it’s not just helpful—it’s impressive. It saves time, reduces stress, and makes you look like the calmest person in the room when systems are on fire. 💡 The best part? It’s free and open-source: 🔗 GitHub: https://lnkd.in/dNjR4kqU 🌐 Website: https://www.brentozar.com/ Huge respect to Brent Ozar Unlimited Brent Ozar and the team for creating something that helps DBAs worldwide. 🙌 Brent, your First Responder Kit has saved my career more times than my backups ever did.” 😅 DBAs—have you tried the First Responder Kit? If yes, what’s the biggest win it’s given you? #SQLServer #DBA #PerformanceTuning #Community #OpenSource

190

16 Comments

To move millions of rows or GBs of data from one SQL Server to another, try using Copy-DbaDbTableDataCopy from dbatools! I ran a quick demo comparing two commands: 🔹 Write-DbaDbTableData 🔹 Copy-DbaDbTableData Both use SQL Bulk Copy under the hood, but here’s what surprised me: 🔹 Copying 1000 rows: Write-DbaDbTableData: 4.5 seconds Copy-DbaDbTableData: 0.067 second The big difference is because Write-DbaDbTableData buffers data in the memory of the machine running the command, while Copy-DbaDbTableData doesn’t, and that makes it way faster and more efficient for large datasets. Also these functions can auto-create destination tables! Details on how to use these functions - https://lnkd.in/eQ6G6B9T #SQLServer #dbatools #PowerShell #DatabasePerformance #SQLTips #DataEngineering #PerformanceMatters #SQLCommunity

258

37 Comments

Data gets messy long before it gets valuable. That’s why platforms like Databricks matter. Databricks brings data engineering, analytics, and machine learning into a single workspace built on Apache Spark. No juggling tools. No fragile pipelines. Just one place to ingest data, transform it, analyze it, and train models at scale. The Lakehouse approach is the real shift here. You get the flexibility of data lakes with the reliability of data warehouses, powered by Delta Lake. That means ACID transactions, schema enforcement, and the ability to trust your data as it grows. For data teams, this translates to faster iteration, fewer handoffs, and less time spent managing infrastructure. If you’re working with large, complex data and still stitching together too many systems, Databricks is worth understanding. #Databricks #BigData #DataEngineering #Analytics #MachineLearning #Lakehouse

50

You trained the model. You trusted the output. But can you trace the data that led to the decision? 👀 AI in analytics isn’t dangerous because it’s wrong. It’s dangerous because it’s convincing, even when it’s wrong. 📰 This article breaks down the real risks of AI-powered insights and how to design for trust with traceability, governance, and human oversight. Read it here: https://lnkd.in/eKuQnv7J

10

10 rules for data professionals to live by: 1. Think like a business owner 2. Choose clarity over complexity 3. Let your soft skills shine 4. Context is king 5. Obsess over outcomes 6. Know your audience 7. Never compromise on QA 8. Make AI your superpower 9. Don’t be afraid to specialize 10. Never stop learning What else would you add? 👇

152

3 Comments

SQL Challenge (Day-6): Fix the Flawed Schema You're consulting for Retaillytics Inc., an e-commerce startup. They've created a test orders table, but the data types are inconsistent and need to be corrected. #SQLChallenge #DataCleaning #DatabaseDesign #SQLSchema #EcommerceData #TechConsulting #DataTypes #RetailAnalytics #SQLFix #DataEngineering #SQLPractice #LearnSQL #360DigiTMG

57

41 Comments

Struggling with slow queries on massive datasets? Traditional data lakes often fall short on performance, making real-time analytics a challenge. The Databricks Lakehouse combines the best of data lakes and warehouses to power faster, smarter decision-making.  ⏬ Download the whitepaper to learn how to fast-track your modernization to Databricks Lakehouse with LeapLogic: https://lnkd.in/ddBSRhAB #databricks #dataestate #modernization #cloudmigration #datamodernization #whitepaper

58

1 Comment

Databricks expertise has become one of the most sought-after skills in data and analytics. The platform combines data engineering, machine learning, and analytics in ways that traditional tools can't match. But finding contractors with real Databricks experience is incredibly challenging. What we're seeing in the market: ✔️ Senior Databricks engineers commanding premium rates across all sectors. ✔️ Companies struggling to scale data initiatives without specialist platform expertise. ✔️ Contract lengths extending as organisations recognise the learning curve for internal teams. ✔️ Global demand from Europe, the US, and increasingly APAC markets. ✔️ Cross-industry adoption from financial services to retail to manufacturing. ✔️ Integration complexity requiring deep understanding of cloud architectures and data governance. ✔️ The specialist contractors who understand both Databricks technical capabilities and business implementation are writing their own contracts. Investment in Databricks capability isn't optional anymore. It's competitive necessity. Are you seeing similar talent challenges with specialist data platforms? #Databricks #DataEngineering #DataAnalytics #TechnologySpecialists

22

Live coding exercise for my conference talk at Data Day Texas? You bet! Saturday - 2nd Floor - Ballroom 204 Code: The Untapped Metadata Source Driving Most Data Failures Over the past three years, collaborating with organizations implementing data contracts and writing the O'Reilly book on the topic, it has become abundantly clear that data is surprisingly the wrong surface to address systemic data issues —we actually need to focus on code. Code not only defines what data is captured and moved through software systems, but it's also the ultimate source of truth in how a business operates and its documented changes over time. This is evident in how some of the most significant internet outages can be traced not only to code changes, but also to how those changes affected underlying data models. In this talk, I share case studies and lessons learned in extracting metadata from code and using it to enforce expectations across software systems and data products.

23

2 Comments

SSMS 21 Solves a major Blocking Issues!!!  Exciting news for SQL Server DBA's and users ! The team behind SSMS is continuously improving the most popular SQL tool in the world, based on feedback from customers - and this time, it's my turn! DBA's often faced an issue in SSMS when attempting to DROP a database or execute ALTER DATABASE, only to be stopped by idle sessions caused by connection pooling. Despite the GUI providing an option to close active connections when dropping a database, there was no simple way to do this via script, resulting in unexpected blockages. Thanks to Microsoft, this problem has been addressed. The game-changing feature in SSMS 21 is a new option that addresses this issue. Additional features such as incorporating Git and enabling Dark Mode... Download it and experiment with the below link https://lnkd.in/g4JSCbmx

383

27 Comments

Analytics Governance isn’t optional in the Public Sector. It’s urgent. From misconfigured dashboards to manual QA bottlenecks, public agencies face mounting pressure to deliver trustworthy data analytics. In a world where data drives decisions and public trust, governance at the Analytics layer is no longer a nice-to-have. 🔍 Discover the 4 key reasons why Analytics Governance must be part of every public agency’s strategy, and how Wiiisdom helps make it happen. 📖 Read the full article at https://lnkd.in/ehpjvPXD #AnalyticsGovernance #governance #decisionmaking #PublicSector

7

The Data Analyst in Databricks career track is built for beginners, but it goes deep enough to matter on the job. You’ll practice SQL, work with Delta Lake, build dashboards, and learn how analytics actually runs in the Databricks Lakehouse. Along the way, you’ll cover what you need for the Databricks Certified Data Analyst Associate exam! Start for free 👉 https://ow.ly/SaQF50XQA4w

137

5 Comments

Quartr API datasets are now available via Snowflake. Access structured earnings data – live audio, real-time and historical transcripts, filings, slide decks, and event summaries. Pull Quartr data directly into your existing Snowflake workflows and augment your datasets. No new pipelines, no separate API integration. Build with data that drives decisions.

75

💡 Still running your DWH on SQL Server? Time to rethink. If you have: • a team that kills sessions because of locks • ETL jobs only at night so nothing breaks • schema changes that take a week to “coordinate” • fear of DELETE because it might freeze everything …then you’re solving modern problems with outdated tools. Try Apache Iceberg: • ACID transactions • real schema evolution • deletes, merges, time travel • works with Spark, Trino, Flink, Snowflake & more ☁️ In the cloud, it’s faster and way cheaper. Store once in S3/GCS, query with anything. Compute when you need it — no lock-in, no waste. If your “data platform” still looks like SQL Server + nightly jobs + manual babysitting — something’s off. #Iceberg #DataEngineering #ModernDataStack #SQLServer #CloudData #DWH

347

143 Comments