[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Input line too long???
I played with your example and I think I have a solution. The input
buffer limitation is only in effect if you read formatted. I made a file
by repeating your example line 2000 times, obtaining 10000 element line.
Each record in your line is 8 bytes long, including a space. Try reading
unformatted and it works perfectly:
; Make fake data file with 1 line
; with 10000 data entries and open it.
openr, unit, 'test.txt', /get_lun
; Since you know the number of data points, make X:
x = strarr(10000)
; Fill X with 8 byte strings.
x[*] = '12345678'
readu, unit, x
Now you can convert X into float and use it.
David Bowman wrote:
> Greetings all,
> I have a data file that essentially contains a very long vector of
> floating point numbers. The file (written by a trusty old FORTRAN
> program) gives the number of elements in the record on the first line.
> The second line contains all the data, with each element separated by
> two spaces. For example:
> .123348 .126664 .130130 .133735 .137486 and so on.
> Now in the past I've read this data with no problem simply using
> where n is the number of elements in the vector. However, I've
> recently tried to read in a much larger vector, and IDL chokes, saying:
> % Input line is too long for input buffer of 32767 characters.
> The datafile in question contains 5856 elements (five of which are in
> my little example above). All told, there are 55278 characters in the
> line in question.
> Nothing I've tried works... Reading in elements by element, with
> formatting, without formatting... I've even tried using the BUFSIZE
> keyword to OPEN, with no result. The IDL help says that I'm limited to
> reading in files smaller than 2^31 bytes, but I'm certainly well below
> I'd love to simply rewrite the I/O section of the program that made the
> file to begin with, but I didn't write it and I don't have the source
> Any Ideas? I'm using version 5.3 on a Mac G3 with 128 Mb RAM.
> Thanks in advance...
> David Bowman "Smash forehead on keyboard to continue."
> Laboratoire Tectonique
> Institut de Physique du Globe - Paris
> 4 Place Jussieu
> 75252 Paris Cedex 05