Hello, torch.use_deterministic_algorithms
> GPU floating-point operations aren't always perfectly deterministic
What?
GIF
Quote
@cneuralnetworkOct 29
neural nets.
"temperature = 0 means the LLM gives the same output every time"
not necessarily
- temperature 0 makes decoding greedy, picking the highest-probability token
- GPU floating-point operations aren't always perfectly deterministic
- batching and parallel execution can slightly
