Dropped dimensions?

[Craig Markwardt <craigmnet(at)astrog.physics.wisc.edu>]

I really enjoy programming in IDL.  Because of dynamic typing and
dimensioning of variables, and the inherent vector nature of most
operators, the language itself can be exceptionally powerful.  I find
myself doing some very mind-twisting things with ease in IDL, which
become very difficult if I have to translate to C.

However, sometimes IDL simply drives me up the wall.  No surprise,
it's a problem with IDL silently dropping the last dimension of a
vector variable.  Recently we've seen it causing havoc with matrix
multiplication.  Here's another example:

Goal: use TOTAL to total one dimension in an array, A.  A is three
dimensional, but can have any dimensions (ie, it can even be 1x1x1).
Typically I want to total the last dimension of the three.

ATTEMPT 1:   *******************************************
  A = DBLARR(N1, N2, N3)
  ...  processing ...
  TOT = TOTAL(A, 2)   

ANALYSIS: Looks good right?  Wrong, because IDL can silently drop any
number of trailing dimensions of size 1 from the array, so
occasionally the array doesn't have a third dimension to total.  Okay,
we can REFORM it and try again.

ATTEMPT 2:   *******************************************
  A = DBLARR(N1, N2, N3)
  ...  processing ...
  A = REFORM(A, N1, N2, N3, /OVERWRITE)  ; Make sure dimensions are correct
  TOT = TOTAL(A, 2)

ANALYSIS: In fact, you will see this formalism a lot in my code.  I
usually reform an array instinctively after I create it, just to be
sure it has the dimensions I ask for!  Okay but this still has a
problem because sometimes, if A is a 1x1x1 array at the start, the
processing can leave only a scalar.  Surprise again!  Because REFORM()
does not accept scalars.  So this is what I am left with:

ATTEMPT 3:   *******************************************
  A = DBLARR(N1, N2, N3)
  ...  processing ...
  IF N_ELEMENTS(A) EQ 1 THEN A = [A]     ; Make sure it's an array
  A = REFORM(A, N1, N2, N3, /OVERWRITE)  ; Make sure dimensions are correct
  TOT = TOTAL(A, 2)
     
ANALYSIS: Okay, this works in most cases.  But it's a lot of hoops to
jump through for a simple operation.

What is the moral of the story?

For IDL programmers: you have to be very careful about where your
array variables get silently REFORMed.  REFORM them yourself at
critical points.

For RSI:
  * REFORM should operate on scalars too.
  * TOTAL should ignore missing final dimensions, since those dimensions
    can be dropped.
  * Dimensions should not be dropped!  I do appreciate when that happens
    sometimes, but it usually happens at random and dangerous moments.
    I would like to have explicit control over when it happens.  Something
    like a RELAX procedure which "relaxes" unneeded final dimensions.

Thanks for staying with me on this tirade.  Back to shiny happy
thoughts now.

Craig


-- 
--------------------------------------------------------------------------
Craig B. Markwardt, Ph.D.         EMAIL: craigmnet@astrog.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
--------------------------------------------------------------------------