[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Plot bug or another "strange feature" ?
- Subject: Re: Plot bug or another "strange feature" ?
- From: Paul van Delst <pvandelst(at)ncep.noaa.gov>
- Date: Tue, 16 May 2000 11:12:22 -0400
- Newsgroups: comp.lang.idl-pvwave
- Organization: NOAA/NWS/NCEP/EMC
- References: <39216B47.32808BD0@Noveltis.fr>
- Xref: news.doit.wisc.edu comp.lang.idl-pvwave:19618
Nicolas Decoster wrote:
> Does anybody notice and, eventually, know how to handle this "strange
> feature" of the plot procedure ?
> IDL> s = indgen(1000) + 1000000000
> IDL> plot, s, /ynozero
> The plot is not a clean line joining down-left and up-right corners, but
> an ugly stairway...
I thought that sort of thing came about because:
a) PLOT converts all its arguments to single precision floats. Do a
PRINT, FLOAT(s[0:100]),FORMAT='(e20.13)' to see what happens to the
b) people think they can represent numbers at the extremes of machine
precision exactly. Your "s" array, when converted to single precision
float, changes in the 8th or 9th d.p. While I think that IDL should
allow users to set the PLOT conversion to double precision if they want
(e.g. with a DOUBLE keyword or something), in general you can't expect
these sorts of numbers to be represented well - in IDL or any language.
That's not how floating point arithmetic works. All floating point
numbers are approximations to their actual value.
Paul van Delst Ph: (301) 763-8000 x7274
CIMSS @ NOAA/NCEP Fax: (301) 763-8545
Rm.202, 5200 Auth Rd. Email: email@example.com
Camp Springs MD 20746