[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
accuracy problems
- Subject: accuracy problems
- From: "Jörg Schliwa" <joerg.schliwa(at)ewetel.net>
- Date: Thu, 27 Apr 2000 17:24:08 +0200
- Newsgroups: comp.lang.idl-pvwave
- Organization: EWE TEL
- Xref: news.doit.wisc.edu comp.lang.idl-pvwave:19421
HY to all!
I've a very strange problem: I'm running IDL 5.2.1 on a Pentium III 500 with
Windows NT. When I read data from a file into a variable, IDL always makes a
truncation. Even when I type it at the IDL prompt.
For example:
IDL> a=2.83456
IDL> print, a
IDL prints : 2.00000
Its equal if I define the variable as float or double, the result is always
the same.
Does anyone have a idea how this can happen,
Jörg