Float number calcuation....

['Paul Z. Wu' zwu_net-/[e-mail] [seajug]]

What is the result of?

  System.out.println(.0014*100);


I thought JavaScript's behavior was weird, but I was wrong. I guess almost all the programming language behave the same.


 
Paul Z. Wu

[Douglas Pearson doug-list-S/[e-mail] [seajug]]

You mean you were expecting 0.14 instead of 0.13999999999999999?

Yeah that's just normal floating point math. Remember it's binary under
the hood, not decimal so you can't exactly represent all decimal numbers in
a 64-bit binary value.

Doug

Post by 'Paul Z. Wu' zwu_net-/***@public.gmane.org [seajug]
What is the result of?
System.out.println(.0014*100);
I thought JavaScript's behavior was weird, but I was wrong. I guess almost
all the programming language behave the same.
Paul Z. Wu

[Jim White [e-mail] [seajug]]

It says "Computer Scientist" in the title, but this applies to programmers
too:

*What Every Computer Scientist Should Know About Floating-Point Arithmetic*

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

But that excellent work is actually TLDR for most folks, so you'll probably
want to check this out instead:

*What Every Programmer Should Know About Floating-Point Arithmetic*
*or*
*Why don’t my numbers add up?*

http://floating-point-gui.de/

Jim

Post by 'Paul Z. Wu' zwu_net-/***@public.gmane.org [seajug]
What is the result of?
System.out.println(.0014*100);
I thought JavaScript's behavior was weird, but I was wrong. I guess almost
all the programming language behave the same.
Paul Z. Wu

['Paul Z. Wu' zwu_net-/[e-mail] [seajug]]

Yes, I knew something about this even when I was in school many years ago -- like doing  number comparison involving float numbers is tricky. This was still a small surprise until I saw it after users complained.

 
 
Paul Z. Wu




________________________________
From: "Jim White james.paul.white-***@public.gmane.org [seajug]" <***@yahoogroups.com>
To: "seajug-***@public.gmane.org" <seajug-***@public.gmane.org>
Sent: Tuesday, July 8, 2014 1:58 AM
Subject: Re: [seajug] Float number calcuation....



 
It says "Computer Scientist" in the title, but this applies to programmers too:

What Every Computer Scientist Should Know About Floating-Point Arithmetic

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html


But that excellent work is actually TLDR for most folks, so you'll probably want to check this out instead:

What Every Programmer Should Know About Floating-Point Arithmetic
or
Why don’t my numbers add up?

http://floating-point-gui.de/


Jim

Post by 'Paul Z. Wu' zwu_net-/***@public.gmane.org [seajug]
 
What is the result of?
  System.out.println(.0014*100);
I thought JavaScript's behavior was weird, but I was wrong. I guess almost all the programming language behave the same.
 
Paul Z. Wu
 

[Eric Jain [e-mail] [seajug]]

Post by 'Paul Z. Wu' zwu_net-/***@public.gmane.org [seajug]
What is the result of?
System.out.println(.0014*100);

The result is platform-dependent (unless you are you using the
strictfp keyword).
--
Eric Jain
Got data? Get answers at zenobase.com.


------------------------------------
Posted by: Eric Jain <eric.jain-***@public.gmane.org>
------------------------------------


------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/seajug/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/seajug/join
(Yahoo! ID required)

<*> To change settings via email:
seajug-digest-***@public.gmane.org
seajug-fullfeatured-***@public.gmane.org

<*> To unsubscribe from this group, send an email to:
seajug-unsubscribe-***@public.gmane.org

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/