Lucas Beyer (bl16) on X: "I like to think that i know reasonably well what to expect from floats on gpus, but the consistency in this one still surprised me!"

I like to think that i know reasonably well what to expect from floats on gpus, but the consistency in this one still surprised me!
1/4 New writeup: you can fingerprint the exact GPU you're running on with a default torch.index_add_ call, even among identical GPUs. This works because floating-point maths depends on order, and every GPU has its own. This fingerprint survives reboots and can't be faked.