JavaScript Number Weirdness

Well, that looks like you've run in to the limits of using floating point. Essentially, floats have limited precision, so sometimes you'll technically have very minute imprecisions in either direction. This has some implications on comparisons in edge cases when using floats.

Other apps round numbers more thoroughly when outputting them, so you never notice.

IOW, JS is doing things correctly, but QC outputs numbers in a non-standard way. Internally, this same representation is probably used whether the JS engine runs within a browser or QC. The difference is the method used to get a string representation. Incidentally, how does the number look if you Log() it?

sing it with me: Floating Point is Not Exact, and ALWAYS has rounding taking place. Never Exact, Never Exact, Never Exact. (chorus)

er... anyway: You cannot store 0.6 in Floating point.

here's a simple test program:

#include <stdio.h>
 
int main()
{
   double frac = 0.6;
   unsigned int i;
   unsigned char *bytes = (unsigned char*)&frac;
 
   for(i=0;i<8;++i)
      printf("%02x", bytes[i]);
   printf("\n");
 
   printf("frac: %20.20f\n", frac);
 
   return 0;
}

Basically, we're writing the actual 64bit value that is used to encode 0.6 in Floating point (for curiosity), and then printing the number with as much precision as we can (20 decimal places above and below the decimal point -- 64bit can only handle ~15-16 decimal places, so we get to watch the edge cases where precision falls off).

What do you expect the output to be?

here's what it is:

333333333333e33f
frac: 0.59999999999999997780

Wait, what? 0.5999999blahblah ? The program clearly has 0.6?! Unfortunately, 64bit floating point Can't represent 0.6 perfectly. It can represent 0.5 perfectly. Same with 0.25, and other various numbers. But not 3/5. Because it's a repeating number (in binary), it has to chop off some precision once it hits the end of the 64bits allotted to doubles. There's simply no more room to add the infinity bits required to store 0.6 as a decimal. (if you worked with rational fractions, you can do it by storing the numerator and the denominator separately, and there are libraries to help you do this, but it's orders of magnitude slower than using hardware floating point stuff).

So sorry, but you cannot store 0.6 (and other various fractions). You'll always have some error down towards the bottom, and there's nothing you can do to correct it. Usually, when displaying the number to the user, you'll chop off anything beyond 6-8 decimal places, just to keep that from being visible.

Noticed similar thing the other day. Is it restricted to JS though? Wasn't there discussion on QC-list the other week about double precision and Cocoa. If you happen to have a gmail account then this link works: http://mail.google.com/mail/#search/rounding+issues+in+cocoa

Apple archive wont return a hit for the same search (or any other appropriate token for that matter) Neither will google using 'site:lists.aplle.com' token :/ Search has to be the flakiest technology on the web these days. Gimme a couple of million bucks and I know how to solve it!

Mr Laird, one of my school Maths teachers said I was smart but not intelligent for writing this on the blackboard (on account of him often emphasising this point).

Nope, it's computer specific ;)

Math with real numbers is nearly always done this way in any form of computing, since it's fast and good enough for most things. If you need more precision, there are other, more slow and unwieldy ways, eg. fixed point. You have to go out of your way to use those, and would probably only do it if you had a really good reason.

Consider that you've been using floats this long without noticing. If the problems is just the readability of the numbers, there are ways around that. Use a string, not a number port for anything you send to an Image with String, for instance.

Thanks guys, for the explanation- and sorry for my ignorance of basic computing concepts. I guess like a lot of people, I've just picked up what I needed to know as I went along, rather than ever really sitting down and going back to brass tacks (as it were).

It's not a problem, actually, works fine- I was just curious.

a|x

sbn..'s right on here -- it's always been this way.

Here's the mailing list thread you were referring to: http://lists.apple.com/archives/Quartzcomposer-dev/2009//Sep/msg00216.html

This quirk has been known to programmers for decades, but it's not widely discussed outside of programming circles because for the most part non-programmers are very much shielded from this detail. It's quite interesting to see QC exposing non-programming folks to this kind of stuff -- exciting for me, but probably annoying for them :/

cwright wrote:
It's quite interesting to see QC exposing non-programming folks to this kind of stuff -- exciting for me, but probably annoying for them :/

I don't mind being exposed to all this kind of stuff. I don't expect to absorb it all, though... ;)

I am however, very grateful to all you programming types for your patient explanations.

a|x

at leaste your not counting money. thats how i first encountered this problem in a java class making a virtual atm machine, that returns coins. i just added the difference instead of rounding. kind of makes me think of the movie "office space"

i think this problem actually cause a space ship to crash as well that is why nasa is trying to make computers write code. apparently this is a human problem, and the hypothesis is that if computers wrote their own machine language it would be much more precise. at least thats the theory.

For exact, non-fractional units (or with fixed precision, like currency), integer or fixed precision is the way to go, because they're exact.

the problem of limited precision is fundamental to CPUs -- it's not a human problem, except that humans haven't invented an infinite-precision calculator.

With a 64bit floating point number, it can only store so much precision -- that has nothing to do with humans. a 4-digit number in base 10 can only store 0-9999, regardless of whether it was made by humans, computers, aliens, dogs, or anything. It's a property of Information Theory.

Computers already write their own machine language -- C is high level, and the compiler will translate it to assembly, and then to machine code. Along the way, a bunch of changes take place (for optimization, usually, though there are other uses for transformation). You can write machine code by hand if you want, but that's a pretty exotic job these days -- virus and malware authors will do that for shellcode and exploits, and JIT-compiler authors can do it, but that's about it. Most professional developers I'm aware of will write only in C/C++/ObjC, and a small handful can drop to assembly if necessary, in small doses (I did a bit of asm work for the upcoming Kineme3D 1.2 release, for example), but even that's pretty scarce.

There are libraries like GMP that make more precision code very easy to write (I've used that for cryptographic exploration a few years back), it's just a matter of using the correct tool for the job.

we had this guy and girl come to campus one day for a silicon valley talk or something. i guess the lady was the programmer for the first lap top or first lap top in space and the guy ditrech i think his name was worked for a bunch of corporations. i guess they where into the guts of assembly seeing the two met each other while working for tandy non stop computers. you know the ones that banks use or whatever where you can take an axe to the thing and it will not stop computing. anyways to make a long story short this guy says he buries his money and doesn't trust the digital banking systems. im all but you wrote the code for these systems and you don't trust it and he replies i am only "human"

toneburst wrote:
Thanks guys, for the explanation- and sorry for my ignorance of basic computing concepts. I guess like a lot of people, I've just picked up what I needed to know as I went along, rather than ever really sitting down and going back to brass tacks (as it were).

It's not a problem, actually, works fine- I was just curious.

a|x

Heh, I'm the same in terms of just picking up what I need. I just happened to notice this before and was lectured by some CS types, and so read a bit up on it.

So, just passing on the lecture ;)

cwright wrote:
Here's the mailing list thread you were referring to: http://lists.apple.com/archives/Quartzcomposer-dev/2009//Sep/msg00216.html

Interested how you managed to generate this hit/link cwright. What is it that I'm missing here?

I found it manually -- lists.apple.com/archives/Quartzcomposer-dev/2009/Sep/ gave me the list, and I searched (cmd-f in firefox) for "rounding" to find the thread :)