[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: rounding errors



Alex Schuster wrote:
> You just have to define the 2.56989 as a double: b = 2.56989d :)
> 
> Remember that the floating point value 2.56989 is not the same as the
> real number 2.56989000000000000..., but just some value around
> 2.5698900. Converting it to a double introduces more digits to the
> right, but they are pretty random.
> 
>         Alex
> --
>   Alex Schuster     Wonko@weird.cologne.de          PGP Key available
>                     alex@pet.mpin-koeln.mpg.de

Danke Alex :)

  the prints were more of an example. In the 'real' world, 
  the data is read in from a file with readu and is an array
  of float variables. I want to perform the following maths 
  in double but with the cast to double I already introduce 
  'pretty random digits to the right', as you say.
  I'ld really like to avoid calling something like double(string(a))
  for some large array...

Any ideas?

Schönes langes Wochenende,
       Dominic

[ Tuesday is a holiday, so I'll take Monday off :) ]
-- 
Dipl. Phys. Dominic R. Scales | Aero-Sensing Radarsysteme GmbH
Tel:  +49 (0)8153-90 88 90    | c/o DLR Oberpfaffenhofen
Fax:  +49 (0)8153-908 700     | 82234 Wessling, Germany
WWW:  aerosensing.de          | email: Dominic.Scales@aerosensing.de