"temperature = 0 means the LLM gives the same output every time"
not necessarily
- temperature 0 makes decoding greedy, picking the highest-probability token
- GPU floating-point operations aren't always perfectly deterministic
- batching and parallel execution can slightly change numerical results
- even tiny differences can flip the next token when probabilities are close
and once one token changes, the entire response can diverge. deterministic decoding ≠ deterministic inference
- "temperature = 0 means the LLM gives the same output every time" not necessarily - temperature 0 makes decoding greedy, picking the highest-probability token - GPU floating-point operations aren't always perfectly deterministic - batching and parallel execution can slightly change numerical results - even tiny differences can flip the next token when probabilities are close and once one token changes, the entire response can diverge. deterministic decoding ≠ deterministic inference
check them out :)growing up watching Harry Potter, i always wanted a room full of wizards who could do things for me. turns out we're getting pretty close to that with AI agents. @getskydive just launched a 3D virtual office where you can build your own team of AI coworkers. each agent gets its
Wasn’t there a 2025 paper from thinking machines that proved that LLMs are deterministic?
