“The only scalable way to measure complex metrics like hallucination, conciseness is by using another LLM tuned and trained to evaluate your model’s responses.” That was Oguzhan (Ouz) Gencoglu Gencoglu, Co-Founder & Head of AI at Root Signals, on this week’s episode of the (AI) People podcast. We explored why traditional MLOps metrics can’t capture what really matters in LLM-powered products. Ouz believes LLM-as-a-Judge is not just useful it’s the only way to get reliable, scalable evaluation without relying on armies of humans. 👉 Would you trust an LLM to judge another LLM’s output?
Steve Dodsworth
6mo
hmm but who's judging the judge tho? seems like we're just kicking the trust problem up one level..
Ouz Gencoglu
6mo
Thanks for having me! Enjoyed our chat Ben Jackson.
https://open.spotify.com/episode/6z10hajoddFB9mVwTFZoRW?si=aGnM2IfZTEapSKQwvo4PUA