Re: Reading tab delimited string data

[Martin Schultz <martin.schultz(at)dkrz.de>]

Andy Loughe wrote:
> 
> Hello,
> 
> I have some output from a MySQL dump that looks like this...
> 
> id      date    height  coverage        yy      yn      ny      nn
> 1388    2000-07-01      all     all     114     404     240     28452
> 1388    2000-07-01      all     low     114     404     240     28452
> 1388    2000-07-01      high    all     114     404     240     28452
> 1388    2000-07-01      all     all     114     404     240     28452
> 
> I get the data into IDL via this command...
> 
> spawn, 'mysql < sql_script.sql', data
> IDL> help, data
> DATA       STRING  = Array[5]  ; 5 rows of string data (tabs inserted).
> 
> Basically, I am working with tab delimited data with a varying number
> of columns and a varying number of rows, depending upon the sql query.
> 
> I am able to store the data from each column into a separate IDL vector,
> id=[ ]  date=[ ]  height=[ ] , etc. but the approach is to LOOP
> through all rows, do a str_sep(row, string(9B)), and then direct the
> result into the proper index of the IDL vector.
> 
> Looping through all the rows and manipulating each line of string data
> can't be the fastest approach.  Can someone suggest a faster method?
> 
> I thought of ASSOC, and also wondered about str_sep on arrays,
> or maybe some of the new str routines.  Any thoughts?
> 
> Your kind suggestions are appreciated.
> 
> --
> Andrew Loughe =====================================================
> NOAA/OAR/FSL/AD   R/FS5  |  email: loughe@fsl.noaa.gov
> 325 Broadway             |  wwweb: www-ad.fsl.noaa.gov/users/loughe
> Boulder, CO  80305-3328  |  phone: 303-497-6211   fax: 303-497-6301


off the top of my head I can only think of using structures. You can
dynamically create a template structure (just as you set up your
column vectors) with Create_Struct, e.g. template =
Create_Struct(id:0L, 'date','', 'height','', 'coverage','', $
             'yy',0L, 'yn',0L, 'ny',0L, 'nn',0L )

Then you have two options:
(1) if you don't know the number of rows coming out of your query
(well, this applies more to results stored in a file), you can either
use a linecount program and use approach 2 afterwards, or you still
loop line by line and use a "block data" technique to enlarge your
structure array by e.g. 100 elements at a time.

(2) if you know how many lines to expect, you should be able to "read"
your data as
    result = replicate(template, 5)
    reads, data, result

Hope this helps,
Martin

-- 
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
[[ Dr. Martin Schultz   Max-Planck-Institut fuer Meteorologie    [[
[[                      Bundesstr. 55, 20146 Hamburg             [[
[[                      phone: +49 40 41173-308                  [[
[[                      fax:   +49 40 41173-298                  [[
[[ martin.schultz@dkrz.de                                        [[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[