Heather Barnhart’s Post

AI+DFIR 2026 Challenge: The Good vs The Ugly To enable data-driven discussions about GenAI in investigations, Brian Carrier is organizing a 4-week challenge with a panel of judges (AI advocates and skeptics), public voting, and sharing all of the results. The goal here is not to promote or bash any single product or LLM. It’s to share what currently works and what doesn’t. The basic concept is: You submit SANITIZED screen shots of where GenAI was amazing, where it went bad, and where you’re not sure it helped or hurt. A panel of industry judges will review for the top 5 amazing ones and the top 5 disasters. The public will vote on the final winners. The winners get bragging rights! The judges: Me - Heather Barnhart (SANS) Alexis Brignoni (LEAPPS) Eric Capuano (Digital Defense Institute) Brian Carrier (Sleuth Kit Labs - Organizer) Filip Stojkovski (BlinkOps) Submissions: Submissions are due by May 25, 2026 11:59PM EST. The form is here: https://tally.so/r/vG0rrQ Submission Requirements The goal here is for honest and well intentioned submissions from practitioners using data from:  Actual investigations CTFs Course data sets Realistic test data Vendors can submit results from their own tools, but they need to disclose they are a vendor! Example public data sources include: https://lnkd.in/ekTnj4pg https://ctf.null404.org https://cfreds.nist.gov/ Submissions will include: Context of the data What you prompted Screenshots of the results Why do you think it’s amazing, a disaster, or a snooze-fest? The criteria will include: Clarity: Is it obvious from the screenshot + context what happened? Can someone learn from it without needing a 10-minute explanation? Significance: Did the result provide either a much faster result or a novel finding? Or a really dangerous finding? Realistic: Is the data set realistic or is it a bit esoteric?  Teachability: Would this make someone better at using (or being skeptical of) GenAI in their workflow? Is there a takeaway from it? Requirements to Win Submit all of the info on the form (screen shots, context, etc). Make sure to include your email so that we can verify it’s a real submission. We won’t publish this though and results can be posted anonymously. Schedule May 25: submissions are due June 8: Public voting begins June 15: Public voting ends June 18: Winners are announced If you have any questions, send them to Brian Carrier.

We need more BAD submissions. If you have seen GenAI do a bad job show us! This is also a learning exercise.

To view or add a comment, sign in

Explore content categories