[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

*Subject*: Re: rounding errors*From*: James Kuyper <kuyper(at)wizard.net>*Date*: Wed, 02 May 2001 11:07:23 -0400*Newsgroups*: comp.lang.idl-pvwave*Organization*: Not enough*References*: <3AE9330C.29D059E9@aerosensing.de> <3AE94F58.45E4BA83@pet.mpin-koeln.mpg.de> <3AE95A3F.D32172CA@aerosensing.de> <Pine.LNX.4.21.0104271343370.13163-100000@mulligan.atm.ox.ac.uk> <3AE9805E.B60C084@ssec.wisc.edu> <Pine.LNX.4.21.0104271545290.13163-100000@mulligan.atm.ox.ac.uk> <3AE99476.170D84E6@noaa.gov>*Xref*: news.doit.wisc.edu comp.lang.idl-pvwave:24778

Paul van Delst wrote: > > Randall Skelton wrote: > > > > On Fri, 27 Apr 2001, Liam E. Gumley wrote: > > > > > This is a subtle but important point. DOUBLE() is a type conversion > > > function, and > > > > > > a = double(2.348339) > > > > > > shows a FLOAT argument being converted to a DOUBLE. The safest way to > > > 'cast' a double variable is > > > > > > a = 2.348339d > > [snip] > > > > Wow... I am glad that I have now learned that particular 'IDL feature' > > early on in my PhD. Just yesterday, I convinced the department that we > > really need a few good IDL programming books as the current > > 'learning-by-fire' approach could have some unfortunate consequences ;) > > This "feature" has absolutely *nothing* to do with IDL. The same thing occurs in other > languages, e.g. Fortran, C, etc. Floating point numbers, in general, cannot be represented > exactly and you have to keep that in mind when writing code I think you're misunderstanding the "feature" of IDL that surprised me as much as it surprised Randall and Liam. This has everything to do with IDL, and nothing to do with expecting exact representation of a finite-length decimal fraction. By default, in C 2.348339 represents a double precision number, not a single precision one, and I'd never realized that the IDL convention was different. That probably explains a number of the wierd results I've seen.

**Follow-Ups**:**Re: rounding errors***From:*Paul van Delst

**References**:**rounding errors***From:*Dominic R. Scales

**Re: rounding errors***From:*Alex Schuster

**Re: rounding errors***From:*Dominic R. Scales

**Re: rounding errors***From:*Randall Skelton

**Re: rounding errors***From:*Liam E. Gumley

**Re: rounding errors***From:*Randall Skelton

**Re: rounding errors***From:*Paul van Delst

- Prev by Date:
**Re: rounding errors** - Next by Date:
**Re: Image overlay** - Prev by thread:
**Re: rounding errors** - Next by thread:
**Re: rounding errors** - Index(es):