Thom Ives, Ph.D.

Great survey on small language models. This 87-page survey closely examines small language models, defined as models sized between the minimum threshold for emergent abilities on specialized tasks and the maximum sustainable under resource constraints. SLMs are really worth keeping a close eye on. They are specifically useful in multi-agent systems.

168

6 Comments

Just read a fun new post from Benjamin Rogojan (Seattle Data Guy) on why data teams shouldn’t blindly sprint toward Apache Iceberg. As always, Ben’s writing is provocative in the best way - challenging us to pause and ask why we’re reaching for the latest tech before we commit. I'm going to counter Ben on this one: Firsthand, I’d argue that there is a clear place for it. One where it not only accelerates time-to-insight, but also helps reduce the need to pile more junk onto the dump. In the analytics world, we’re constantly under pressure to answer questions fast. Stakeholders needed insights yesterday, so we chase whatever gets us to a solution quickest. Most of the time, those swift moves leave behind a heap of tech debt to clean up next year. But is Iceberg just another patch for deeper infrastructure issues …or does it actually have a practical home in your stack? Take 6sense: in 2022 calendar year alone, we went from two distinct warehouse environments to 5+. Why? M&A of 4 product companies and numerous new feature launches with evolving architecture needs.... But more environments don’t change expectations. Everyone still wants centralized analytics. A real question just two weeks after acquisition of company #3: “Can we just see how usage of feature X in warehouse A + adoption of capability Y in environment B impacts customer health stored in warehouse C?” Sounds easy, right? Just write a cross-environment query and wrap it up before lunch? In reality, you’re scrambling to stitch everything together, suddenly managing dozens (or hundreds) of EL jobs just to centralize data. At our peak, we had over 100 of these running nightly - each one draining compute, delaying other analytics pipelines, and straining SLA expectations. Would Iceberg have solved all of that? No. Integrating acquired systems and legacy stacks is never simple. But had we adopted Iceberg earlier - with strong governance and a cross-functional design approach that prioritized analytics from the start - we could’ve avoided a lot of duplication and orchestration overhead. Iceberg is NOT a silver bullet. But in the right context, it’s a powerful architectural lever for data democratization and it can absolutely drive both efficiency and cost savings. Curious to hear from others: Has your team adopted Iceberg? What made it worth it…or not? Highly recommend giving Ben a follow if you work in data. The post in reference in comments!

29

19 Comments

In June, I gave the keynote at Cascadia R Conference 2025: "25 Things You Didn't Know You Could Do with R." R is often seen as just a stats tool, but I showed how it’s also great for: ✔️ pulling data from Google Sheets, Qualtrics, and the Census Bureau ✔️ data visualization ✔️ mapping and geospatial analysis with sf and ggplot ✔️ automating reports with GitHub Actions ✔️ integrating AI tools like GitHub Copilot The conference just posted the talk. Give it a watch and see all the resources I shared: https://lnkd.in/ejEscFY4 I hope that you learn something new. #rstats

122

3 Comments

Myself and James Brand have been chipping away at `dbreg`, an #rstats 📦 for running big regression models on database backends. For the right kinds of problems, the speed-ups are often incredible. Website: https://lnkd.in/gKBb8vAR New features include: - fast binscatter regression - new acceleration strategies - analytical clustered SEs - interaction terms support - stats methods (predict, vcov, etc.) - integration with tabling sw - etc. Full v0.0.3 changelog: https://lnkd.in/gXwtU8rP

104

3 Comments

Resource-Effective Analyst Relations A new report about analyst relations went live on October 16, 2025. Here's an outline ... Analyst relations (AR) managers need optimally efficient plans because their teams are generally smaller than they feel they need. So how can AR optimize its processes, spending plans, and resource deployment without spending excessive time analyzing all the relevant efficiency factors? This report presents a technique that draws on information already in your plan, supplemented by simple ratings, to give you a visual picture of resource-effective AR practices that’s based on your unique AR objectives and performance targets. Forrester clients can find the report at https://lnkd.in/eSjXZfjG.

49

2 Comments

How do you know if your new A/B testing methodology actually works? Researchers continue to propose new ways to analyze A/B tests: new estimators of impacts, new rules for deciding whether to launch. But it's difficult to tell whether these methodologies actually add value because the true impact of any A/B test is unobserved. When they give us different results, are they actually better --- or just different? Traditionally, we'd turn to simulations. But this requires strong assumptions about the data generating process that are unlikely to hold in practice. My new paper (with James McQueen and Miikka Rokkanen) attempts to solve this problem via sample splitting. The idea is to partition each test into two independent splits: one to proxy for the true impact and another to apply the proposed methodology. We show exactly what this approach can and cannot measure, how to conduct statistical inference, and the tradeoffs involved in implementation. We hope this will help practitioners rigorously evaluate whether new methodologies are actually adding value in their A/B testing contexts. Any and all feedback is welcome! https://lnkd.in/g5ZvCbkv

272

6 Comments

"Data modeling is often invisible work, and invisible work rarely gets funded." The latest draft chapter, "How Politics and Power Influence Data Models" from my upcoming data modeling book just dropped on Substack. Read it here: https://lnkd.in/gQSVHQNH

521

34 Comments

One of the most common concerns I hear from data leaders who are looking to go down the graph path is the fear that this will necessitate *yet another* massive lift & shift operation. The answer is no, and I explain the reason here: https://lnkd.in/gSnSWpcu Central to this question is: what kinds of data go into the graph, what is the "minimum viable graph"), and can I grow my graph over time. Foreshadowing the answers: you don't need everything, and yes graph databases--the native ones, i.e. not backed by an RDBMS-- are literally designed for organic expansion and yielding network effects.

89

8 Comments

Data literacy is growing - graphs are mainstream. Which is why articles like this are SO important. We need to know what makes for great data visualization so we can apply those criteria to graphs and determine what’s actually just a marketing picture. https://lnkd.in/gzmtaJWX

17

1 Comment

In Big Data, almost any T-Test of means returns significant. Have you ever had that problem? If you have two samples from Big Data and you test the difference of means, they will probably be statistically significant. But that does not mean they're relevant. What can you do to know if the difference is relevant? ** Try Cohen’s d. The Effect Size Every Data Scientist Should Know ✔️ P-values tell you whether an effect exists. ✅ Cohen’s d tells you how big that effect is. It measures the standardized difference between two group means — perfect for A/B tests, experiments, medical studies, or any comparison task. How to interpret Cohen’s d: 0.2 → Small effect 0.5 → Medium effect 0.8+ → Large effect * Why it matters: ✔️ Avoids the trap of over-relying on p-values ✔️ Helps stakeholders understand practical significance ✔️ Great for experiment prioritization and A/B test decisions ✔️ Effect size > statistical significance. 👉 Learn more in my recent post: https://lnkd.in/emsJk6vG #Statistics #ABTesting #Experimentation #DataScience #MachineLearning #CohensD #Python #Analytics

19

3 Comments

AI agents are dominating the 2025 tech landscape with so many options available. This post breaks down the strengths and weaknesses of the some of the widely used agent frameworks so that others can make informed decisions Key Takeaways: ► OpenAI Agents SDK – Best for simplicity and production-readiness, with a low learning curve and built-in primitives like Agents, Handoffs, and Guardrails for structured workflows. ► LangGraph – Ideal for complex, graph-based workflows but has a steeper learning curve. ► CrewAI – Most intuitive for role-based multi-agent systems, but lacks scalability and is still evolving. ► AutoGen – Supports diverse conversation patterns and agent topologies, but documentation is unclear. Link: https://lnkd.in/gr6pVhJ2

13