[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: rounding errors
Alex Schuster wrote:
> You just have to define the 2.56989 as a double: b = 2.56989d :)
> Remember that the floating point value 2.56989 is not the same as the
> real number 2.56989000000000000..., but just some value around
> 2.5698900. Converting it to a double introduces more digits to the
> right, but they are pretty random.
> Alex Schuster Wonko@weird.cologne.de PGP Key available
Danke Alex :)
the prints were more of an example. In the 'real' world,
the data is read in from a file with readu and is an array
of float variables. I want to perform the following maths
in double but with the cast to double I already introduce
'pretty random digits to the right', as you say.
I'ld really like to avoid calling something like double(string(a))
for some large array...
Schönes langes Wochenende,
[ Tuesday is a holiday, so I'll take Monday off :) ]
Dipl. Phys. Dominic R. Scales | Aero-Sensing Radarsysteme GmbH
Tel: +49 (0)8153-90 88 90 | c/o DLR Oberpfaffenhofen
Fax: +49 (0)8153-908 700 | 82234 Wessling, Germany
WWW: aerosensing.de | email: Dominic.Scales@aerosensing.de