There have been several questions posted to SO about floating-point representation. For example, the decimal number 0.1 doesn’t have an exact binary representation, so it’s dangerous to use the == operator to compare it to another floating-point number. I understand the principles behind floating-point representation. What I don’t understand is why, from a mathematical perspective, … Read more
How do I generate random floats in C++? I thought I could take the integer rand and divide it by something, would that be adequate enough? 14 Answers 14
Why does changing the sum order returns a different result? 23.53 + 5.88 + 17.64 = 47.05 23.53 + 17.64 + 5.88 = 47.050000000000004 Both Java and JavaScript return the same results. I understand that, due to the way floating point numbers are represented in binary, some rational numbers (like 1/3 – 0.333333…) cannot be … Read more
Sometimes, Activerecord data types confuse me. Err, often. One of my eternal questions is, for a given case, Should I use :decimal or :float? I’ve often come across this link, ActiveRecord: :decimal vs :float?, but the answers aren’t quite clear enough for me to be certain: I’ve seen many threads where people recommend flat out … Read more
I’m working with map data, and the Latitude/Longitude extends to 8 decimal places. For example: Latitude 40.71727401 Longitude -74.00898606 I saw in the Google document which uses: lat FLOAT( 10, 6 ) NOT NULL, lng FLOAT( 10, 6 ) NOT NULL however, their decimal places only go to 6. Should I use FLOAT(10, 8) or … Read more
I have a Decimal(‘3.9’) as part of an object, and wish to encode this to a JSON string which should look like {‘x’: 3.9}. I don’t care about precision on the client side, so a float is fine. Is there a good way to serialize this? JSONDecoder doesn’t accept Decimal objects, and converting to a … Read more
I keep seeing this constant pop up in various graphics header files 0.0039215689 It seems to have something to do with color maybe? Here is the first hit on Google: void RDP_G_SETFOGCOLOR(void) { Gfx.FogColor.R = _SHIFTR(w1, 24, 8) * 0.0039215689f; Gfx.FogColor.G = _SHIFTR(w1, 16, 8) * 0.0039215689f; Gfx.FogColor.B = _SHIFTR(w1, 8, 8) * 0.0039215689f; Gfx.FogColor.A … Read more
I have float numbers like 3.2 and 1.6. I need to separate the number into the integer and decimal part. For example, a value of 3.2 would be split into two numbers, i.e. 3 and 0.2 Getting the integer portion is easy: n = Math.floor(n); But I am having trouble getting the decimal portion. I … Read more
I’ve been working with data imported from a CSV. Pandas changed some columns to float, so now the numbers in these columns get displayed as floating points! However, I need them to be displayed as integers or without comma. Is there a way to convert them to integers or not display the comma? 11 Answers … Read more
Why do comparisons of NaN values behave differently from all other values? That is, all comparisons with the operators ==, <=, >=, <, > where one or both values is NaN returns false, contrary to the behaviour of all other values. I suppose this simplifies numerical computations in some way, but I couldn’t find an … Read more