CUDA's non-determinism is in scheduling and memory access, not computation itself. This is the same guarantees as you get with a CPU; the computation is deterministic, but the point at which you work ...
Despite being a CEO, I'm an introvert, and some of the most joyful experiences of my life have been locked away inventing algorithms. Software development mostly consists of messy glue and data pipes ...
GPUs do provide the same IEEE 754 guarantees on determinism when it comes to floats as CPUs do; they're fully deterministic machines in that regard. The reason people talk about GPUs as ...