Skip to content

Conversation

@Alvaro-Kothe
Copy link

@Alvaro-Kothe Alvaro-Kothe commented Jan 31, 2026

Rationale for this change

This PR restores the behavior previous to version 23 for floating-point parsing on overflow and subnormal.

fast_float didn't assign an error code on overflow in version 3.10.1 and assigned ±Inf on overflow and 0.0 on subnormal. With the update to version 8.1, it started to assign std::errc::result_out_of_range in such cases.

What changes are included in this PR?

Ignores std::errc::result_out_of_range and produce ±Inf / 0.0 as appropriate instead of failing the conversion.

Are these changes tested?

Yes. Created tests for overflow with positive and negative signed mantissa, and also created tests for subnormal, all of them for binary{16,32,64}.

Are there any user-facing changes?

It's a user facing change. The CSV reader on version libarrow==23 was assigning them as strings, while before it was parsing it as 0 or +- inf.

With this patch, the CSV reader in PyArrow outputs:

>>> import pyarrow
>>> import pyarrow.csv
>>> import io
>>> table = pyarrow.csv.read_csv(io.BytesIO(f"data\n10E-617\n10E617\n-10E617".encode()))
>>> print(table)
pyarrow.Table
data: double
----
data: [[0,inf,-inf]]

Closes #49003

@github-actions
Copy link

⚠️ GitHub issue #49003 has been automatically assigned in GitHub to PR creator.

Copy link
Contributor

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm @pitrou

const auto res =
::arrow_vendored::fast_float::from_chars_advanced(s, s + length, *out, options);
return res.ec == std::errc() && res.ptr == s + length;
return (res.ec == std::errc() || res.ec == std::errc::result_out_of_range) &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit on readability - I think it would help to break up this line. Something like:

const auto is_allowed_ec = res.ec == std::errc() || res.ec == std::errc::result_out_of_range;
return is_allowed_ec && res.ptr = s + length;
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely it's better. I also took the liberty to store the pointer comparison in an auxiliary variable in favor of readability.

@Alvaro-Kothe Alvaro-Kothe force-pushed the fix/behavior-change-float-underflow branch from 9166e21 to 163c538 Compare January 31, 2026 22:17
@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Feb 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2 participants