floating-point Archives

floating-point, Programming

IT Nursery

What’s the difference between a single precision and double precision floating point operation?

What is the difference between a single precision floating point operation and double precision floating operation? I’m especially interested in practical terms in ...

June 4, 2022
0 Comments

performance, Programming

IT Nursery

What does gcc’s ffast-math actually do?

I understand gcc’s --ffast-math flag can greatly increase speed for float ops, and goes outside of IEEE standards, but I can’t seem to ...

June 2, 2022
0 Comments

Programming, types

IT Nursery

Which is the first integer that an IEEE 754 float is incapable of representing exactly?

For clarity, if I’m using a language that implements IEE 754 floats and I declare: float f0 = 0.f; float f1 = 1.f; ...

June 2, 2022
0 Comments

Programming, Python

IT Nursery

How to display a float with two decimal places?

I have a function taking float arguments (generally integers or decimals with one significant digit), and I need to output the values in ...

June 1, 2022
0 Comments

Programming, Python

IT Nursery

How to suppress scientific notation when printing float values?

Here’s my code: x = 1.0 y = 100000.0 print x/y My quotient displays as 1.00000e-05. Is there any way to suppress scientific ...

May 31, 2022
0 Comments

Java, Programming

IT Nursery

Meaning of delta or epsilon argument of assertEquals for double values

I have a question about JUnit assertEquals to test double values. Reading the API doc I can see: @Deprecated public static void assertEquals(double ...

May 29, 2022
0 Comments

bash, Programming

IT Nursery

How can I compare two floating point numbers in Bash?

I am trying hard to compare two floating point numbers within a Bash script. I have two variables, e.g. let num1=3.17648e-22 let num2=1.5 ...

May 29, 2022
0 Comments

C++, Programming

IT Nursery

Compare double to zero using epsilon

Today, I was looking through some C++ code (written by somebody else) and found this section: double someValue = ... if (someValue < ...

May 27, 2022
0 Comments

Using bitwise OR 0 to floor a number

A colleague of mine stumbled upon a method to floor float numbers using a bitwise or: var a = 13.6 | 0; //a ...

May 26, 2022
0 Comments

Format Float to n decimal places

I need to format a float to “n”decimal places. was trying to BigDecimal, but the return value is not correct… public static float ...

May 25, 2022
0 Comments