What is the difference between a single precision floating point operation and double precision floating operation? I’m especially interested in practical terms in relation to video game consoles. For example, does the Nintendo 64 have a 64 bit processor and if it does then would that mean it was capable of double precision floating point … Read more
I understand gcc’s –ffast-math flag can greatly increase speed for float ops, and goes outside of IEEE standards, but I can’t seem to find information on what is really happening when it’s on. Can anyone please explain some of the details and maybe give a clear example of how something would change if the flag … Read more
For clarity, if I’m using a language that implements IEE 754 floats and I declare: float f0 = 0.f; float f1 = 1.f; …and then print them back out, I’ll get 0.0000 and 1.0000 – exactly. But IEEE 754 isn’t capable of representing all the numbers along the real line. Close to zero, the ‘gaps’ … Read more
I have a function taking float arguments (generally integers or decimals with one significant digit), and I need to output the values in a string with two decimal places (5 → 5.00, 5.5 → 5.50, etc). How can I do this in Python? 12 Answers 12
Here’s my code: x = 1.0 y = 100000.0 print x/y My quotient displays as 1.00000e-05. Is there any way to suppress scientific notation and make it display as 0.00001? I’m going to use the result as a string. 16 Answers 16
I have a question about JUnit assertEquals to test double values. Reading the API doc I can see: @Deprecated public static void assertEquals(double expected, double actual) Deprecated. Use assertEquals(double expected, double actual, double delta) instead. (Note: in older documentation versions, the delta parameter is called epsilon) What does the delta (or epsilon) parameter mean? 8 … Read more
I am trying hard to compare two floating point numbers within a Bash script. I have two variables, e.g. let num1=3.17648e-22 let num2=1.5 Now, I just want do a simple comparison of these two numbers: st=`echo “$num1 < $num2” | bc` if [ $st -eq 1]; then echo -e “$num1 < $num2” else echo -e … Read more
Today, I was looking through some C++ code (written by somebody else) and found this section: double someValue = … if (someValue < std::numeric_limits<double>::epsilon() && someValue > -std::numeric_limits<double>::epsilon()) { someValue = 0.0; } I’m trying to figure out whether this even makes sense. The documentation for epsilon() says: The function returns the difference between 1 … Read more
A colleague of mine stumbled upon a method to floor float numbers using a bitwise or: var a = 13.6 | 0; //a == 13 We were talking about it and wondering a few things. How does it work? Our theory was that using such an operator casts the number to an integer, thus removing … Read more
I need to format a float to “n”decimal places. was trying to BigDecimal, but the return value is not correct… public static float Redondear(float pNumero, int pCantidadDecimales) { // the function is call with the values Redondear(625.3f, 2) BigDecimal value = new BigDecimal(pNumero); value = value.setScale(pCantidadDecimales, RoundingMode.HALF_EVEN); // here the value is correct (625.30) return … Read more
