ppc64le - Fix deviation in floating point values caused by FMA #91
Conversation
|
@rsned, please review. |
|
@otan, fyi |
|
@ambujkumar3 fyi |
|
Can you see if this still occurs with the latest updates. |
|
Yes. Cockroachdb still encounters FMA failures with the latest golang/geo. |
This was exposed while testing CockroachDB. Following tests from pkg/sql/opt/exec/execbuilder fail due to floating point precision differences caused by FMA: - TestExecBuild/local/geospatial - TestExecBuild/local-vec-off/geospatial - TestExecBuild/local-spec-planning/geospatial - TestExecBuild/fakedist/geospatial - TestExecBuild/fakedist-vec-off/geospatial - TestExecBuild/fakedist-metadata/geospatial - TestExecBuild/fakedist-disk/geospatial - TestExecBuild/fakedist-spec-planning/geospatial With explicit casts in this patch, these failures are resolved. References: https://go.dev/ref/spec#Floating_point_operators golang/go#53297 golang/go#48145 disintegration/gift#20 (comment)
jmr
force-pushed
the
ppc64le-cdb-execbuilder
branch
from
153cf68 to
79b2dce
Compare
|
Does go have anything like https://learn.microsoft.com/en-us/cpp/preprocessor/fp-contract?view=msvc-170 It seems we'll be playing whack-a-mole otherwise. There is |
|
We've done some whack-a-mole previously which did resolve some of them. I know nothing about fiddling with compiler optimization settings in Go. This one is simple enough to add, but I would say any future ones would need more convincing or figuring out the right set of Go build flags to embed in the file. |
|
This change makes the code more portable across platforms without dependency on the compiler. |
|
I didn't find any go option to disable FMA, so I guess this is as good as it gets. |
3 participants
This was exposed while testing CockroachDB.
Following tests from pkg/sql/opt/exec/execbuilder fail due to floating point precision differences caused by FMA:
With explicit casts in this patch, these failures are resolved.
References:
https://go.dev/ref/spec#Floating_point_operators
golang/go#53297
golang/go#48145
disintegration/gift#20 (comment)