Thursday, April 2, 2015

Geek: 7 bits of mantissa are not enough for two (decimal) digit accuracy...

Geek: 7 bits of mantissa are not enough for two (decimal) digit accuracy...  Excellent discussion of this little-understood problem, with a (for me, at least) new perspective of the underlying issue as one of interleaving.  Browsing around the site led to all sorts of interesting posts...

No comments:

Post a Comment