[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Conversion Error?
- Subject: Re: Conversion Error?
- From: m218003(at)modell3-d.dkrz.de (Martin Schultz)
- Date: 16 Aug 1999 14:42:40 GMT
- Newsgroups: comp.lang.idl-pvwave
- Organization: Deutsches Klimarechenzentrum, Hamburg
- References: <9STt3.65114$jl.41007316@newscontent-01.sprint.ca> <MPG.1221be1dffece505989893@news.frii.com>
- Xref: news.doit.wisc.edu comp.lang.idl-pvwave:16132
>> IDL> size=(double(200-70.1))/0.01
>> IDL> help,size
>> SIZE DOUBLE = 12990.000
>> IDL> c=fltarr(size)
>> IDL> help,c
>>
>> I get
>>
>> C FLOAT = Array[12989]
>>
>> Does anyone know why???
That might be in the FAQ by now? Anyway: try
print,(double(200-70.1))/0.01,format='(f24.14)'
and you'll see what you really get.
Solution: add a tiny little bit to your double number before truncating it
by using it as an index (LONG please, not INTEGER !). To determine how much
to add, you should compute the ALOG10 and subtract about 9 or 10 from it.
In your case that would give you something like -5 which means you would
add 10^-5 = 0.0001 .. hardly noticeable but efficient in getting correct
numbers. **BUT**: as any speed fanatics will tell you: logs and likes are
very costly in computer resources, so whenever you know the numerical range
of values use a constant rather than the computation above.
Regards,
Martin.