[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

*Subject*: Re: Set Precision???????????*From*: Craig Markwardt <craigmnet(at)cow.physics.wisc.edu>*Date*: 14 Jun 1999 11:18:11 -0500*cc*: Anil Kochhar <anilk(at)mtolympus.ari.net>*Newsgroups*: comp.lang.idl-pvwave*Organization*: U. Wisc. Madison Physics -- Compact Objects*References*: <Pine.BSF.4.05.9906141118510.13580-100000@mtolympus.ari.net>*Reply-To*: craigmnet(at)cow.physics.wisc.edu*Xref*: news.doit.wisc.edu comp.lang.idl-pvwave:15182

Anil Kochhar <anilk@mtolympus.ari.net> writes: > > Hi All > > I'd like create a varible which always holds numbers out to 6 places > passed the decimal point, and 4 numbers before the decimal point (e.g. > 1999.123456). I tried double > precision but this seems to only hold 8 digits total for decimal numbers. > However I would like to create a variable corresonding to a fraction of a > Year (e.g 1995.123456). I have not been able to find a procedure allowing > me to create 10 digit decimal number , without the number being rounded > off to 8 digits. > Does anyone know of a way to declare a variable to hold a 10 digit > decimal number? You should make a distinction between the *internal precision* of a floating point or double precision number, and the *textual representation* of that number. Both are quite different. Internally, double precision maintains about 16 decimal digits in the mantissa irrespective of the position of the decimal point, which is plenty more than you need. [ Floating point keeps about 7 decimal digits, probably not enough for you. ] The important thing to keep in mind is that IDL doesn't always print this many digits when it prints a number. I'm sure this is primarily for convenience, since people probably don't want huge multi-digit output on their screen most of the time. By *default*, IDL prints 8 digits before the decimal, and 7 after the decimal (at least it did in my simple trial). If you want to have a different output format, then you need to tell IDL explicitly how to format it using the FORMAT keyword to PRINT, PRINTF or STRING. That's too complicated a subject to write about here but I can get you started. FORMAT strings are similar in spirit to IDL format statement in FORTRAN, or the format string in C. For double precision output formats, you describe (a) the total record length, and (b) the number of digits after the decimal. In your case, the total record length is 11 ( = 4 year digits + 1 decimal point digit + 6 fractional digits), and the number of digits after the decimal is 6. The format string is thus 'D11.6'. Here is how that goes together in an IDL print statement. IDL> y = 1999.123456D IDL> print, y 1999.1235 IDL> print, y, format='(D11.6)' 1999.123456 Voila! Craig -- -------------------------------------------------------------------------- Craig B. Markwardt, Ph.D. EMAIL: craigmnet@astrog.physics.wisc.edu Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response --------------------------------------------------------------------------

**References**:**Set Precision???????????***From:*Anil Kochhar

- Prev by Date:
**Set Precision???????????** - Next by Date:
**POLYWARP question.** - Prev by thread:
**Set Precision???????????** - Next by thread:
**POLYWARP question.** - Index(es):