Ashish Gyanchandani

Most data teams don’t have a data problem - they have a pipeline problem. Every new source adds complexity, slows teams down, and creates more to maintain. At one51, we’re solving this using the latest capabilities in Snowflake. Our Data Onboarding Framework makes ingestion simple, scalable, and reliable: → Deploy in a single call → Works across AWS, Azure & GCP → Adapts to schema changes automatically → Built-in security and governance The impact? Less time building pipelines. More time delivering value from data. Read the full article to see how it works https://lnkd.in/gTimEXs7 #DataEngineering #Snowflake #ModernDataStack #CloudData #Analytics Bhanupriya S

9

Host of “Truth wit Tuesdays 🚀” (Goes live Every Tues 9:30 AM IST) As a seasoned Data science professional , i believe in simplifying data & its usage in AI ML through real life use cases We have build ourselves a chain of learning from #Datalineage #DataAudit #DataCompleteness #DataStory #DataCreators #DataQualityFrameworks #ProblemSolving Today we speak of "Challenges" that are current hurdle for any organization to adopt Gen AI fully , we will cover technical, ethical, operational and human factors 1.Data Quality & Governance We know the quality of Advanced AI relies on quality of data And that is currently bad, inconsistent , siloed , divided by data privacy , compliance concerns and ownership and provenance issues 2. Talent & Skill Gaps Training budget and time lag behind tech adoption rates are not in sync 3. Integration with Legacy systems Significant engineering efforts will be required to build the "foundation" first Currently Monolithic architectures resist AI components APIs , data pipelines and real time systems aren't easily ready 4.Regulation & Compliance uncertainty The legal landscape is moving too fast with no global standards for AI governance Liabilities &penalties can hit unknowingly 5.Cost & ROI Ambiguity Deploying AI isn't cheap , cloud compute , talent , model , no clear ROI In the end the question still lingers "What measurable value are we getting"? 6. Ethics , bias , societal concerns Advanced AI can amplify problems if misused (biased outputs , misinformation, lack of ethical guardrails) Ethical AI governance is the need of the hour 7.Maturity of AI strategy This part is the main devil where not having clear roadmap leads you to dark hole /burnout as Org Some close observation i see and hear is: 🔍 Treating GenAI like a toy or pilot not as strategic asset 🔍Confusing experimentation with deployment 🔍Failing to connect use cases to business outcomes

14

1 Comment

Tredence’s recognition as a Databricks Gold Partner marks a significant step forward in our partnership with Databricks, reflecting our ongoing commitment to helping organizations operationalize data and AI at scale. This milestone is rooted in our ability to move beyond architecture into last-mile execution, designing and delivering real-world data and AI solutions that drive measurable impact across the enterprise. In this blog, Jason Y. shares the story behind this journey and the principles that guide how we deliver, anchored in our BEACON values and a strong focus on customer success. As organizations continue to invest in data intelligence, we remain focused on enabling scalable, future-ready data and AI capabilities that drive innovation and growth. Proud of what this milestone represents, and even more excited about what lies ahead with Databricks. Read the full story: https://lnkd.in/gfmWHBzB #Tredence #BeyondPossible #Databricks

210

4 Comments

Over the past couple of years, I’ve worked extensively with large language models. In machine learning, it’s often said that nearly 80% of the effort goes into curating high-quality data, with the remaining 20% spent on training the model. That has largely been my experience as well. I’ve spent countless hours curating datasets and training models so they could perform well on internal benchmarks. After the hardest parts - curating the data and training the model are behind you, it’s tempting to consider the work mostly done. In practice, however, this is only the beginning. Models rarely improve in isolation. Meaningful progress depends on real user feedback, and that requires making the model accessible to a large number of users. Deploying a model at scale introduces an entirely different set of challenges, ones that are often more complex than training the model itself. Beyond state-of-the-art architectures and modelling techniques, what truly enables adoption is a reliable and scalable system around the model. When building such systems, we have to keep two major constraints in mind - - Users should not have to wait long for a response - Overall latency must remain low enough for the experience to feel seamless. I have explored these challenges in a two-part article series, focusing on the system-level considerations required to serve LLMs reliably and efficiently at scale. Below, I’m sharing the first article in the series. https://lnkd.in/gwNah5uu Thanks for taking out time to read this, and stay tuned for the second article which will be dropping soon!

89

2 Comments

Our portfolio company, TileDB , announced a partnership with Databricks to harness multimodal data to power AI in healthcare and life sciences. https://lnkd.in/g9Z56Bwk

6

The original intent of starting this podcast was a forum to teach. But within a few episodes, I realized it was actually a much higher-leverage resource for accelerating my own learning. In today’s episode of the Data Neighbor podcast, our guests are taking the day off. Hai, Sravya, and I use the time to look back on what exactly we were thinking when we started this thing a year ago, everything we’ve learned along the way, and how this podcast has connected us to so many people we never would’ve met otherwise. We talk about the stuff we did not expect at all (both the good parts and the parts that are way more work than they look from the outside). Honestly, this one is just a really fun episode. Only downside: you have to listen to me ramble a little more than the pleasure of our much more well-spoken guests. Still a good time, and a nice way to wrap up the year. omg is this Data Neighbor Wrapped?! Check it out in the comments ;)

6

1 Comment

Model optimization isn't just about shaving off milliseconds- it's about eliminating friction. No more guesswork. No more stops. Just smooth, intentional performance. In this deep dive, Nityanand Mathur- our inhouse data scientist reveals how to make your inference pipeline feel less like a daily commute… and more like cruise control. Featuring: torch.compile(), CUDA Graphs, dynamic shape handling, memory hacks, and kernel fusion with real code and real results. Link in the comments below for the full blog!

48

4 Comments

Thinking about Apache Iceberg™? Already on Databricks or planning to migrate? Our CTO Eddie Edgeworth, explores three paths to success with Apache Iceberg™ on Databricks, and why open formats paired with open platforms are the future. Read the full blog: https://lnkd.in/gAU7s8em #DataGovernance #Databricks #DataLakehouse #DataInnovation #Koantek #UnityCatalog #DataTransformation #AIInnovation

34

1 Comment

Struggling with your Data Engineering prep? Here’s a bit of help. 🤝 We know how tough it can be to break into product-based companies. That’s why we asked Nishchay, a Senior Data Engineer (ex-Morgan Stanley, Meesho, and now Walmart), to share what he’s learned from clearing 50+ tech interviews. His simple roadmap: Focus on the "why" before the "how." Build projects that solve real-world problems. Don't skip mock interviews, they are a game-changer. Watch the entire video following the link in the comments. We hope these tips help you feel more confident in your journey!

5

1 Comment

🚀 Target Multiplai 2025: AI with Purpose, Scale, and Empathy Just returned from an incredible day at Target’s AI Conference in Bengaluru, and I’m truly inspired by how AI is shaping real-world impact across industries and communities. 🔹 Agentic AI Is Evolving Brad Thompson (SVP, Target Tech) shared frameworks for scaling multi-agent systems, highlighting the importance of balancing autonomy with centralized control—making AI more responsive and practical for enterprise use. 🔹 AI at Scale for a Billion+ Jigar Halani (NVIDIA) showcased India’s use of real-time AI for citizen services—addressing ethics, personalization, and ultra-low-latency decision-making for massive populations. 🔹 Empathy-Driven Innovation Manu Chopra (Karya) delivered a powerful reminder that inclusive AI can create real economic impact. His work is enabling rural communities through AI-powered job opportunities and building more representative datasets. 🔹 Retail’s AI Transformation Target’s real-time pricing engine, built by Rohit Punnoose & Shamally Chopra, is a game-changer—optimizing billions of pricing decisions by aligning competitiveness, customer value, and business growth. ✨ From architecture to accessibility, every session reflected one truth: AI’s future is not just intelligent—it’s inclusive, intentional, and human-centered. #AI #TargetMultiplai2025 #AgenticAI #EthicalAI #RetailTech #InclusiveInnovation #Databricks #NVIDIA #Karya #RealTimeAI #DataScience #MachineLearning

8

1 Comment

Tredence’s Agentic AI Playbook How do enterprises scale AI beyond pilots? Tredence launches its Agentic AI Playbook, making agents peers (not assistants) in workflows. A structured guide for AI maturity & real business value. #AI #EnterpriseAI #AgenticAI https://lnkd.in/gSWj25ey

5

Interview Question Series: Conde Nast – Senior/Lead Data Scientist I’m back with the latest post in my ongoing interview question series, where I share real questions, strategies, and stories from my journey in AI and data science. My aim is to help others preparing for similar roles and to connect with fellow professionals in the field. Company: Conde Nast Position: Senior/Lead Data Scientist Duration: 1 hour 🔹 Key Interview Questions & My Insights 1. What is the bias-variance tradeoff in machine learning? How do you balance underfitting and overfitting? 2. When working with imbalanced datasets, which metrics do you use to evaluate model performance and why? 3. What are the different techniques to determine feature importance in a model? 4. How would you compare Decision Trees, Random Forests, and XGBoost in terms of performance and use cases? 5. Does using a Random Forest increase or reduce variance error? Why? 6. Why does LightGBM often perform better than other algorithms? What features contribute to its effectiveness? 7. What are the key parameters and hyperparameters you tune in a Random Forest model? 8. How do you interpret model results using precision, recall, and F1-score? 9. Why should we use F1-score, and in which use cases is it especially important? 10. What does log loss measure, and why is it important for evaluating probabilistic models? 11. Can you explain the structure of a neural network and the different types of activation functions used? 12. How does forward and backwards propagation work in neural networks? 13. During backpropagation, are biases adjusted along with weights? How does this process work? 14. What is the learning rate, and what are the different types of optimizers? How do they impact model training? 15. What is the vanishing gradient problem, and how does it affect deep neural networks? 16. What is the global minima in the context of training neural networks? 17. How do you determine the number of hidden layers and activation functions in a simple neural network? How many weights, biases, and learning rates are involved in the training process? 🐍 Python Coding Round Given the list `a = [2, 6, 7, 4, 1, 2] write code to count how many times each unique number appears in the list? 💡 Strategies: Storytelling: Instead of just listing my skills, I focused on telling stories—how I solved real problems, learned from mistakes, and grew as a professional. If you found this helpful, please like, comment, or share so it reaches more people. Questions or want to connect? My DMs are open—and for personalized guidance, you can book a session with me on Topmate as well! #InterviewSeries #DataScience #MachineLearning #NeuralNetworks #Python #CareerGrowth #JobHunting #AICommunity #DataScience #SQL #Python #DataScienceInterview

41

3 Comments

#Databricks This course has broadened my perspective on AI strategy and implementation! I took this course to gain an understanding of how Databricks helps build GenAI solutions at scale as a platform. Initially, I hesitated to take this "Fundamentals" course, given my extensive experience in building AI solutions at scale. However, it offered invaluable insights that are often overlooked. I highly recommend it to everyone—whether you're a Leader or a Developer! 📚 What stood out to me 🔍 STRATEGIC ROADMAP FOR AI ADOPTION IN BUSINESS: - Identify GenAI Strategy: Engage business units and define success criteria, while considering legal and ethical policies - Business Use Case: Research and prioritize high-value use cases, ensuring data availability and alignment - Design and Architecture: Choose the right model and integrate it into business systems - Operations & Monitoring: Monitor performance and gather feedback - People and Adoption: Foster a culture of AI acceptance and understanding ⚖️ LEGAL ISSUES : Address data privacy, security, and copyright infringement 🤖 ETHICAL CONSIDERATIONS : Tackle bias and hallucinations in AI 🌍 ENERGY USE AND ENVIRONMENTAL IMPACT : Consider sustainability in AI operations 🔍 RELIABILITY & ACCURACY : Ensure the dependability of AI systems, and Implement robust auditing processes #Databricks #GenAI #AIStrategy #AI

11

4 Comments

I benchmarked 18 forecasting models — from LightGBM to foundation AI. A 1970s algorithm nearly won. The score gap was 0.03. The operational gap was enormous. https://lnkd.in/dTfRUxrB

12

Reflecting on Target Multiplai 2025: Where AI Meets Real-World Impact Had an inspiring day at Target’s AI conference in Bengaluru! Here are my top takeaways: 🔹 Agentic AI is Here to Stay Brad Thompson (SVP, Target Tech) emphasized pragmatic frameworks for scaling multi-agent systems, tackling integration challenges while balancing autonomy and control. 🔹 Real-Time AI for Billions Jigar Halani (NVIDIA) showcased how India leverages AI for hyper-personalized citizen services—navigating ethics, scalability, and real-time decision-making at unprecedented scale. 🔹 Inclusive AI Drives Systemic Change Manu Chopra (Karya) highlighted empathy-driven design—using AI to create economic pathways for rural communities while ensuring datasets represent diverse voices. 🔹 Retail’s AI Revolution Target’s real-time pricing engine (powered by Rohit Punnoose & Shamally Chopra) demonstrated how AI balances competitiveness, profit, and customer value across billions of dynamic decisions. Grateful for the deep dives into ethical scaling, emerging architectures, and AI’s power to uplift. The future isn’t just automated—it’s intelligent, inclusive, and intentional. 🙌 #TargetMultiplai2025 #AI #GenerativeAI #MachineLearning #TechInnovation

77

1 Comment

Vantage now supports billing data ingestion from Databricks System Tables, for the most accurate and native way to visualize your Databricks costs. Previously, before Databricks released System Tables, we ingested cost data from Billable Usage Logs, which provided SKU-level usage at list prices, but didn’t include discounts. That meant users had to do some manual work to reconcile their actual costs. With the new System Tables integration, users can get even more precise usage quantities and see the list price before and after discounts, with no manual work. This makes it easier than ever to visualize and optimize Databricks costs alongside 20+ other native integrations.

32

1 Comment