Re: Plot bug or another "strange feature" ?

[Paul van Delst <pvandelst(at)ncep.noaa.gov>]

Nicolas Decoster wrote:
> 
> Hi.
> 
> Does anybody notice and, eventually, know how to handle this "strange
> feature" of the plot procedure ?
> 
> IDL> s = indgen(1000) + 1000000000
> IDL> plot, s, /ynozero
> 
> The plot is not a clean line joining down-left and up-right corners, but
> an ugly stairway...

I thought that sort of thing came about because:

a) PLOT converts all its arguments to single precision floats. Do a
PRINT, FLOAT(s[0:100]),FORMAT='(e20.13)' to see what happens to the
float'd integers,

b) people think they can represent numbers at the extremes of machine
precision exactly. Your "s" array, when converted to single precision
float, changes in the 8th or 9th d.p. While I think that IDL should
allow users to set the PLOT conversion to double precision if they want
(e.g. with a DOUBLE keyword or something), in general you can't expect
these sorts of numbers to be represented well - in IDL or any language.
That's not how floating point arithmetic works. All floating point
numbers are approximations to their actual value.

paulv
-- 
Paul van Delst           Ph:  (301) 763-8000 x7274 
CIMSS @ NOAA/NCEP        Fax: (301) 763-8545
Rm.202, 5200 Auth Rd.    Email: pvandelst@ncep.noaa.gov
Camp Springs MD 20746