3,237 questions
4029
votes
35
answers
619k
views
Is floating-point math broken?
Consider the following code:
0.1 + 0.2 == 0.3 -> false
0.1 + 0.2 -> 0.30000000000000004
Why do these inaccuracies happen?
- 45.9k
317
votes
7
answers
72k
views
Why are these numbers not equal?
The following code is obviously wrong. What's the problem?
i <- 0.1
i <- i + 0.05
i
## [1] 0.15
if(i==0.15) cat("i equals 0.15") else cat("i does not equal 0.15")
## i does not equal 0.15
- 5,453
265
votes
5
answers
99k
views
Why are floating point numbers inaccurate?
Why do some numbers lose accuracy when stored as floating point numbers?
For example, the decimal number 9.2 can be expressed exactly as a ratio of two decimal integers (92/10), both of which can be ...
853
votes
46
answers
786k
views
How can I deal with floating point number precision in JavaScript? [duplicate]
I have the following dummy test script:
function test() {
var x = 0.1 * 0.2;
document.write(x);
}
test();
This will print the result 0.020000000000000004 while it should just print 0....
- 33.1k
1273
votes
16
answers
465k
views
Why not use Double or Float to represent currency?
I've always been told never to represent money with double or float types, and this time I pose the question to you: why?
I'm sure there is a very good reason, I simply do not know what it is.
- 18.7k
2777
votes
33
answers
4.9m
views
How do I parse a string to a float or int?
How can I convert an str to a float?
"545.2222" -> 545.2222
Or an str to a int?
"31" -> 31
For the reverse, see Convert integer to string in Python and Converting a float to ...
- 69.8k
75
votes
5
answers
26k
views
Why is 24.0000 not equal to 24.0000 in MATLAB?
I am writing a program where I need to delete duplicate points stored in a matrix. The problem is that when it comes to check whether those points are in the matrix, MATLAB can't recognize them in the ...
- 8,554
678
votes
32
answers
691k
views
How do you compare float and double while accounting for precision loss?
What would be the most efficient way to compare two double or two float values?
Simply doing this is not correct:
bool CompareDoubles1 (double A, double B)
{
return A == B;
}
But something like:
...
- 6,821
29
votes
7
answers
136k
views
Floating point inaccuracy examples
How do you explain floating point inaccuracy to fresh programmers and laymen who still think computers are infinitely wise and accurate?
Do you have a favourite example or anecdote which seems to get ...
2455
votes
36
answers
6.1m
views
Limiting floats to two decimal points
I want a to be rounded to 13.95. I tried using round, but I get:
>>> a
13.949999999999999
>>> round(a, 2)
13.949999999999999
For the analogous issue with the standard library Decimal ...
kevin
111
votes
7
answers
290k
views
Division of integers in Java [duplicate]
This is a basic question but I can't find an answer. I've looked into floating point arithmetic and a few other topics but nothing has seemed to address this. I'm sure I just have the wrong ...
- 57.6k
797
votes
11
answers
792k
views
How can I force division to be floating point? Division keeps rounding down to 0?
I have two integer values a and b, but I need their ratio in floating point. I know that a < b and I want to calculate a / b, so if I use integer division I'll always get 0 with a remainder of a.
...
- 129k
2021
votes
41
answers
1.8m
views
How do I check if a string represents a number (float or int)?
How do I check if a string represents a numeric value in Python?
def is_number(s):
try:
float(s)
return True
except ValueError:
return False
The above works, but it ...
- 20.7k
2497
votes
19
answers
1.3m
views
Difference between decimal, float and double in .NET?
What is the difference between decimal, float and double in .NET?
When would someone use one of these?
Tom
94
votes
5
answers
65k
views
Fastest way to do horizontal SSE vector sum (or other reduction)
Given a vector of three (or four) floats. What is the fastest way to sum them?
Is SSE (movaps, shuffle, add, movd) always faster than x87? Are the horizontal-add instructions in SSE3 worth it?
What'...
- 3,858