Quesma develops advanced simulation environments for evaluating and improving agentic AI.
Link zewnętrzny organizacji Quesma
Warsaw, PL
Quesma ponownie to opublikował(a)
Poland was already on the Economist cover, now it is at the top 1 of Hacker News, the homepage of many software engineers. Its economic rise from bankruptcy to the 20th-largest economy is remarkable. I'm inspired and motivated at Quesma to write the next chapter of Poland in the intelligence era.
Quesma ponownie to opublikował(a)
We keep getting visitors at Quesma who raise the bar for how we think. First: Przemyslaw Biecek, leading the Centre for Credible AI at Warsaw University of Technology. We shared perspectives on frontier AI evaluation, credibility, and where the field still needs much better standards. Today: Blazej Osinski, an RL scientist who just spent two years at Altos Labs in Cambridge on what is basically the ultimate moonshot: using ML to help people live much, much longer. Every visit leaves us thinking differently.
Quesma ponownie to opublikował(a)
Stop fearing AI. Start teaching it. That was the theme today at LLMday in Warsaw. Przemysław Hejman and Piotr Migdał (Quesma) stripped away the mystery and got 26 of us tinkering with the Harbor Framework at their workshop. A powerful tool from the authors of Terminal Bench at Laude Institute.
Quesma ponownie to opublikował(a)
Big shoutout to Quesma, Piotr Migdal, Cezary Piwowarczyk, Jacek Migdał and Przemysław Hejman , for partnering with us and helping make this conference happen!🥳 🤖 Quesma tests AI agents in simulation environments that mirror messy, long running tasks instead of staged benchmarks. Teams can see how systems behave over hours, where small errors stack up and shortcuts break. Quesma team is around and they're open to chat about testing AI agents properly and what needs to be in place before you ship them to production. #AIagents #Quesma #LLMdayWarsaw #LLMday
Quesma ponownie to opublikował(a)
How do you test agents instead of guessing? 💻𝗛𝗮𝗻𝗱𝘀 𝗼𝗻 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝘄𝗶𝘁𝗵 𝗛𝗮𝗿𝗯𝗼𝗿 🎙️ Piotr Migdał from Quesma will run a practical workshop on building benchmarks when agent behavior keeps changing. 𝗝𝗼𝗶𝗻 𝘂𝘀 𝗼𝗻 𝗙𝗲𝗯𝗿𝘂𝗮𝗿𝘆 𝟭𝟮 📍𝗖𝗜𝗖 𝗪𝗮𝗿𝘀𝗮𝘄, 𝗣𝗼𝗹𝗮𝗻𝗱 👉 𝗥𝗘𝗚𝗜𝗦𝗧𝗘𝗥: https://lnkd.in/eAiWTKzR 🔥 Piotr builds tools that turn messy data into usable signals.
Quesma ponownie to opublikował(a)
Heading to LLMday Warsaw next week? 🇵🇱 Join me on Feb 12th for a hands-on workshop featuring the Harbor Framework (from Laude Institute). Think of this as your "mental break" from the intensive sessions - a chance to actually build and run agentic LLM tasks applicable to evals, RL, and SFT workflows. 🛠️ Spots are limited - hope to see you in the room! 🤝
Quesma ponownie to opublikował(a)
AI can write code. But can it instrument your code with OpenTelemetry… correctly? Quesma tested 14 top models across 23 OTel instrumentation tasks in 11 languages. The headline result: 14% pass rate. Some spicy details: • Best model hit 29% (Claude Opus 4.5) • GPT-5.2 followed at 26% • Entire task categories are basically unsolved (multiple tasks at 0%) Not like SREs were the most anxious folks to begin with - but this is still much lower than I expected. Link in comments. #OpenTelemetry #OTel #Observability #SRE #DevOps #DistributedTracing #PlatformEngineering #Microservices #AIinEngineering
Quesma ponownie to opublikował(a)
LLMs changed coding fast. But can they do DevOps and SRE? At Quesma, we tested 14 frontier models with basic OpenTelemetry instrumentation. Even the best (Claude Opus 4.5) passed only 29% of tasks. These weren't trick questions, just ~300 lines of code and textbook SRE work. A small subset of typical job responsibilities. Link below