[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Conversion Error?

>>     IDL> size=(double(200-70.1))/0.01
>>     IDL> help,size
>>     SIZE            DOUBLE    =        12990.000
>>     IDL> c=fltarr(size)
>>     IDL> help,c
>> I get
>>     C               FLOAT     = Array[12989]
>> Does anyone know why??? 

That might be in the FAQ by now? Anyway: try 
and you'll see what you really get.

Solution: add a tiny little bit to your double number before truncating it
by using it as an index (LONG please, not INTEGER !). To determine how much
to add, you should compute the ALOG10 and subtract about 9 or 10 from it.
In your case that would give you something like -5 which means you would 
add 10^-5 = 0.0001 .. hardly noticeable but efficient in getting correct 
numbers. **BUT**: as any speed fanatics will tell you: logs and likes are
very costly in computer resources, so whenever you know the numerical range
of values use a constant rather than the computation above.