Steve Fenn

AI is only as good as the context you give it. I've been refining a Tableau chatbot, and my biggest takeaway has been the power of Context Engineering. LLMs have a finite canvas. If you don't manage that space effectively, then you run into issues like context drift and degraded performance. To solve this, I’ve focused on learning ways to maximize the efficiency of the initial tokens passed so no critical insights are lost in the noise. As a first step I have been learning about context engineering in the form of role based tailoring. I have enhanced the chatbot to recognize whether the user is a Viewer, a Manager, or an Executive. - Smart Data Usage: We make the best use of the initial context window to prevent information loss. - Token Efficiency: We aren't just "dumping" data; we are curating the most relevant facts to keep the output sharp and cost effective. - Tailored Insights: The output matches the user's specific expectations and goals. I can now fine-tune these personas in minutes, which historically would have taken days or weeks. The speed at which we can now iterate is a total game changer!

15

In building ML regression models. I’ve always called One Hot Encoding “one hot cookie” as a personal nickname..I forgot and posted my nickname to chat AI tonight.. this was the response 😂😂 “Haha — I love that 😂- sounds like you meant "One-Hot Encoding", not "one hot cookie" ® (though if that existed, I’d want one). Let's walk through exactly how to use One-Hot Encoding in Python with your Income dataset example..”

4

The reality of building Agentic AI in an enterprise environment. This past week at Infogain, I’ve been designing an Agentic AI solution using LangGraph (1 Supervisor + 3 Sub-graphs). The logic was solid, but moving from concept to a stable runtime environment was where the real engineering happened. Here are the 3 biggest hurdles I cleared to get this production-ready: 1. Dependency Hell is real: Balancing langgraph >= 0.2.x with the new OpenAI client and Python versions (3.11 vs 3.12) caused major async execution blocks. Pinned versions are a must. 2. Enterprise Proxies vs. API Calls: The standard OpenAI SDK calls kept hitting corporate proxy walls. Configuring environment overrides for successful passthrough was critical. 3. Concurrency & State: Running sub-graphs in parallel triggered race conditions and serialization errors. I had to restructure the tools to avoid shared-state collisions. The takeaway: Even with powerful LLMs, building reliable AI pipelines still requires deep fundamental engineering and debugging skills. Huge thanks to Vineet Kumar Vivek Gupta for trusting me with this implementation! Excited to keep pushing the boundaries of what Agentic systems can do. #LangGraph #AgenticAI #LLMOps #AIEngineering #Infogain #Python

13

Your complex $1m modern data architecture feels like a legacy system. You are spending more time maintaining it than driving impact. Tightly coupled. Over-engineered. And allergic to change. Every change requires rebuilt. Team spending more time on upkeeping and driving impact. No trust in data or products. Redundant pipelines and tables. More tools than what you see at Gartner Expo. More importantly, no scalability, and another rebuild just around the corner. Meanwhile, the defense sector, the one with more risk, regulation, and bureaucracy than anyone, is doing it differently. They call it data flexibility and being agile. Here is how to build data architecture/infrastructure for scale, impact, and agility. Read full post below: https://lnkd.in/dZ2Gq9C3

3

How I turn vague requests like “what’s going on?” into SQL. Let me paint a familiar picture: A stakeholder pings: “Hey, can you help us understand what’s going on with customer drop-off?” No specifics. No metrics. Just vibes and codes. I used to jump straight into SQL, trying to figure it out. With a bunch of metrics, obviously, I'd almost want to query the whole table. Today? I ask questions first. Here’s how I translate “What’s going on?” into a SQL-ready question: 1. “What part of the customer journey are we focused on?” 2. “What metric are we tracking: logins, purchases, churn?” 3. “What’s the time range we’re comparing?” Then and only then do I query. This tiny shift changed everything. It saved time. Reduced back-and-forth. And helped me get respected for thinking, not just pulling data. If you’ve ever had to turn ‘noise’ into insights, I’d love to hear your method. Drop a tip in the comment section #SQL #DataAnalysis

44

15 Comments

This really does seem a symptom of short term thinking. It might be that some companies when faced with the unpredictable hypothetical issues from the wave of AI, are looking at the one conceptual tool they can think of, "cost-cutting through layoffs"...so they're looking at most possible future issues, as a nail for the layoff hammer.

1

A year ago, if you’d asked me how many decimal places of pi (π) I could memorize, I’d probably laugh. Honestly, I’m not even sure I could confidently recite the first 10 digits. The people close to me know this, my memory has never been the strong suit. That’s why what happened next surprised me. When I started getting interested in neuroscience, part of my attempt to understand human intelligence as inspiration for machine intelligence (spell AI), I kept running into the same idea: memory isn’t just storage. I found that creativity and memory is deeply intertwined. If you learn the right techniques, “good memory” stops being a talent you’re born with and starts becoming a skill you can build. And the interesting part is, by improving creativity, you can improve the memory. With the right approach, memorizing hundreds or even thousands of digits becomes doable and beyond that, it can be extended to learning, acquiring new skills and knowledge, potentially speed up this process in general by several fold. Photo context: This was taken when my Industry Training intern from the Centre for Mathematical Sciences, UMP, was verifying the digits I wrote on the whiteboard. She has a rare talent for spotting patterns in numbers, she was able to memorize them too, but using a completely different approach.

19

1 Comment

Lately I’ve noticed something. Every time I talk with other Tableau admins, the pain isn’t a lack of features. It’s the day-to-day effort of keeping environments sane. So starting Monday, I’m going to try something new. Each week, I’ll share one small admin script or pattern I actually use. Nothing flashy. Just practical ways to make Tableau easier to operate over time. We’ll start with permissions and groups because that’s where complexity quietly piles up. No big plan. Just trying something that feels useful.

16

5 Comments

I love this take, but here’s a bigger question. How do you tackle that debt? How do you “sell” the value of investing time in tech debt/process improvement/general KTLO work. So many different things come at tech teams. New larger projects. Ongoing support. Education/evangalism of exist tools and products. Small projects. Ad hoc requests. I’ve always found prioritization the most difficult part for a data team. Is it just me?

4

If your datasets are poorly labeled, you’re inviting a host of issues like noise, bias, and inconsistencies that can seriously hurt your model's accuracy and reliability when it’s out in the wild. That's why data labeling has evolved from being just a preliminary step into a vital part of engineering and operations in the AI lifecycle.

7

Ever had a moment where you thought, "My AI just ran a ‘simple query’… and the bill jumped"? This scenario underscores the importance of establishing guardrails when agents can perform actions beyond mere conversation. Take this example: You ask, “How many refunds did we have today?” - ✅ Safe way: Use the approved metric table/dashboard, which is fast, cost-effective, and consistent - ❌ Unsafe way: The agent accesses raw transactions, joins six tables, scans a year of data, and retries when it times out, which is slow, costly, and risky (with the potential to access sensitive data) The same question can yield two drastically different outcomes. This is why data platforms need an agent contract: a clear set of rules for agents that includes: - Approved tools/data they can use - The data they must avoid - Limits on timeouts, retries, and spending - Definitions of what 'correct' means (beyond just query succeeded) Prompts are the requests, while contracts act as the seatbelt. For a deeper dive, read the full article in DZone: https://lnkd.in/gWfAbfAD

5

1 Comment

Dashboards don’t build culture, people do. If your data strategy is struggling to stick, it may not be a tech problem. In this episode, I share the mindset shifts and team habits that make data work across housing teams. https://lnkd.in/e5B5cPfh #DataCulture #TeamEngagement #HousingLeadership #renttConsulting #SocialHousing

4

Sparklines can be really helpful to display multiple comparisons minimally. Check out Dan Bunker's take on creating them. Check out the full tutorial from the link below.

30

4 Comments

I used to ask: “WHO should the data leader report to?” Now I know: that’s the wrong question. Here’s the better one: WHAT qualifies a leader to lead the data leader? There’s no universal best answer, but here’s the decision-making lens I use: → Who feels the most pain? Hyper-growth? The CMO may need data most. Profitability phase? CFO’s in the hot seat. → Who’s most qualified? Software Engineering ≠ Data Engineering. Don’t assume CTO is the right fit. → Who has the bandwidth? CEO reporting line can work, but not if they’ve got 10+ directs. → What’s the company’s maturity? Appointing a CDO in a 200-person startup is like showing up in a tux to a BBQ. → What drives the business? Sales-led? Maybe the CMO. Product-led? Maybe the CPO. This org chart decision will either enable your data team - or quietly kill their impact. Who does your data team report to? Drop the job title of your boss in the comments: 👇 👉 And join 3,000+ data leaders who read my free newsletter for weekly tips on building impactful data teams in the AI-era: https://lnkd.in/gRVpJYCV

1,379

92 Comments

I looked into column level lineage on DBT projects and didn't find a free option. What I did find was Claude would vibe code one for me combining the DBT documentation with Elementary testing results for column level lineage with definitions and testing results. I'm new to this, but I made a video about about tracking the lineage of a volatile customer segment of Airbnb guests during a full moon. There is a link to the video and to the project documentation in the comments. #dbt #datalineage #snowflake #analyticsengineering #datagovernance #DCAM

6

2 Comments

Reimagining MLB Playoff Structure: A Data-Driven Approach to Wild Card Allocation I developed an analytical framework that challenges the current playoff system by allocating wild card spots based on division performance rather than fixed league quotas. **The Problem:** Each league currently receives exactly 3 wild card spots, regardless of competitive balance between divisions. **The Solution:** Dynamic allocation based on division strength vs. non-division opponents: • Best division: 2 wild card spots • Second division: 1 wild card spot   • Weakest division: 0 wild card spots **2024 Case Study Results:** - AL East (.562 vs non-division) earned 2 spots, bringing in Boston (81-81) - AL Central (.498 vs non-division) lost representation, removing Detroit (86-76) - NL saw Arizona and Chicago added, while Atlanta and New York were removed **Technical Implementation:** Built using Python with MLB Stats API integration, featuring real-time data processing and interactive Streamlit visualization. The algorithm analyzes 30 teams across 6 divisions, calculating collective performance metrics to determine optimal playoff representation. This system rewards competitive depth while maintaining playoff excitement - a more equitable approach to postseason qualification. https://lnkd.in/gfJq4ziU #BaseballAnalytics #DataScience #MLBPlayoffs #SportsAnalytics

11

Data Team Bingo. Things said in meetings that are technically questions but are actually assignments: "This is probably easy but..." it is never easy, and the word "probably" is doing an enormous amount of heavy lifting. "Can we just add a column for..." there is no such thing as "just adding a column." That column has parents, it has joins, and it has downstream dependencies that will totally fall apart. "I was playing around in the data and..." nothing good has ever followed this sentence. Someone opened a CSV in Excel and now the whole data team is getting a tingly feeling in their left arm. "My numbers are showing something different" your numbers are wrong, but we're going to spend the next three meetings proving that diplomatically. "Can you just pull the raw data so I can take a look myself?" this has never once resulted in fewer meetings. "We should be tracking this" that is a six month project you just proposed like I haven't already been doing it out of spite and a personal vendetta against you. "Can we make it interactive?" no.

97

12 Comments

So Anthony Soltero asked: "What's the most important skill that atrophied that you needed to rebuild?" when I mentioned I made the jump back into data leadership. There's a LOT of skills I needed to brush up on. Of course, I came in with certain technical goals. A lot of process goals. It's a normal trap if you have been an individual contributor. But it's a trap. So, Anthony, the direct answer is: patience. After five years as an individual contributor, my expectation of a successful event horizon was measured in hours or days. I got used to that cadence. That goes completely out the window when you assume a data leadership role. That event horizon shifts to months, or years. Data ecosystems are complex, with many people involved. You're no longer driving a speed boat, you're driving a container ship. It's a whole different game. Once you figure that out, you have a chance at succeeding at the rest.

20

1 Comment

Analytics in mainstream sports is an exhausting place to be. This is a headline from the local Colorado Springs paper. I've seen various renditions of this on different platforms and the commonality is that people think Payton should have kicked a field goal on 4th down in the second quarter. Of course, these takes are easy to make in hindsight. Look at analytics calculators like this one from ESPN: https://lnkd.in/gkftUyEf The percent chance the Broncos had to win if: - They kick the field goal: 67.2% - Go for it : 69.3% Basic math says that's a 2.1% increase in the chance you win the game. And yet... here we are. There are lots of arguments out there against using analytics here. - "It was snowing": No it wasn't. It didn't start snowing until the third quarter. - "Analytics doesn't account for the situation": Yes, it does. That's the whole point. There *are* tangible arguments to make that can counter analytics here. - How prepared is the individual team for a 4th & 1? Do they practice these a lot? - What does their playbook look like for this situation? But the arguments against analytics (and in this article) don't know the answer to that, only the internal team does. Clearly, Payton and the Broncos felt confident in their preparation for this moment. It didn't pan out but that doesn't make it the wrong call. Stop using analytics as a scapegoat when things don't go according to plan. Just because you have a winning percentage over 50 doesn't mean you *will* win, just that it's more likely you will than won't.

30

13 Comments