[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Curious FindFile behavior
- Subject: Re: Curious FindFile behavior
- From: thompson(at)orpheus.nascom.nasa.gov (William Thompson)
- Date: 25 Nov 1998 23:11:20 GMT
- Newsgroups: comp.lang.idl-pvwave
- Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA
- References: <firstname.lastname@example.org> <365C8256.ABBF0F38@attcanada.net>
- Xref: news.doit.wisc.edu comp.lang.idl-pvwave:12972
Dick Jackson <email@example.com> writes:
>Barry Lesht wrote:
>> I'm running IDL5.1 on an SGI (IRIX6.2). I'm using FindFile to create
>> an array of data filenames that I can pass to some processing code.
>> There are (as it turns out) 515 files in the directory I'm searching.
>> The filenames are all of the form xxxxxxxxx.xx.yymmdd.hhmmss.cdf. I
>> get the following from FindFile-
>> IDL> files=FindFile('*.cdf', COUNT=nf)
>> IDL> PRINT, nf
>> IDL> files=FindFile('*980*.cdf', COUNT=nf)
>> IDL> PRINT, nf
>> Can anyone explain this? Thanks.
>I recall seeing cases in IRIX where the result of a system call, if it
>is too large, ends up coming back completely empty instead! From what
>you've said, the result of 'ls *.*' or some such command would be at
>least 15K of text, but less for the last two, as there are fewer files
>to list. There may be a limit in between somewhere. (16KBytes, perhaps?)
The following program, written by a colleague of mine, may overcome this
; Project : SOHO - CDS
; Name : FIND_FILE()
; Purpose : Fixing builtin FINDFILE() problem
; Explanation : The builtin FINDFILE() function has problems on some unixes
; whenever *a lot* of files are matching the file
; specification. This is due to the fact that filename expansion
; is done by the shell *before* interpreting a command. Too many
; files cause too long commands, which are not accepted. This
; causes FINDFILE() to return an empty list of candidates.
; FIND_FILE tries the builtin function first, and whenever the
; returned list of files is empty, it tries to recheck through
; spawning a "find" command.
; Since FINDFILE doesn't discriminate between directories, links
; and files, this function will not do this either.
; Under unix, however, calls like FINDFILE("*") returns the
; unfiltered output of the shell commmand "ls *", including
; colon-terminated lines for each subdirectory matching the
; specification and empty lines separating each subdirectory
; listing. Such silly effects are not implemented in the "find"
; version. Be warned, however, that these effects are present
; when the builtin function does not "fail" due to a too long
; file list.
; It is possible (under unix) to use the "find" method as
; default by setting the keyword /USEFIND (no effect under other
; operating systems).
; Use : files = find_file(file_specification)
; Inputs : file_specification : A scalar string used to find
; files. See FINDFILE()
; Opt. Inputs : None.
; Outputs : Returns a list of files or a blank string if none found.
; Opt. Outputs:
; Keywords : COUNT : Returns the number of files
; USEFIND : Always use a spawned "find" command under unix.
; No effect under other operating systems.
; NODOT : Apply a filter to the results from find to prevent
; finding the directory itself in a large file expansion.
; eg 'find_file,"foo/*"' returns ("foo/","foo/a",...)
; but 'find_file,"foo/*",/nodot' returns
; ("foo/a","foo/b",...) without the leading "foo/".
; This behavior is closer to the behavior of findfile()
; without the long-directory braindamage. It is
; *not* the default so as not to break heritage
; code that uses find_file().
; Calls : BREAD_FILE, FINDFILE, SPAWN
; Common : None
; Restrictions: As for FINDFILE
; Side effects: None, hopefully
; Category : Utilities, Operating_system
; Prev. Hist. : Lots of problems with FINDFILE is hopefully history.
; Written : S.V.H. Haugan, UiO, 12 April 1996
; Modified : Version 2, SVHH, 10 June 1996
; Moved the CD,curr_path command to avoid
; returns without resetting path.
; Version 3, SVHH, 26 June 1996
; Took away the -type f argument to find, added
; /USEFIND keyword.
; : Added /nodot keyword C. DeForest 9-August-1998
; Version : 3, 26 June 1996
count = 0
use_find = KEYWORD_SET(usefind) AND os_family() EQ 'unix'
IF NOT use_find AND N_PARAMS() EQ 0 THEN BEGIN
result = findfile(count = count)
RETURN,result ; Unix doesn't have problems with this
IF N_PARAMS() EQ 0 THEN file_specification = '*'
IF file_specification EQ '' THEN file_specification = '*'
IF NOT use_find THEN result = findfile(file_specification,count=count) $
ELSE count = 0
;; Check for problems
IF count EQ 0 AND os_family() EQ 'unix' THEN BEGIN
file = file_specification
;; Check if directory exists
IF dir NE '' THEN BEGIN
IF (findfile(dir))(0) eq '' THEN RETURN,''
;; Temporary switch to that directory
IF dir NE '' THEN cd,dir,current=curr_path
IF filnam+ext EQ '' THEN filnam = '*'
;; Find all matching
spn = ["find",".","-name",filnam+ext,"-print"]
;; Switch back to original directory
IF dir NE '' THEN cd,curr_path
IF result(0) EQ '' THEN RETURN,'' ; None matching, return
;; Get rid of current-directory match, if necessary
if keyword_set(nodot) and result(0) eq '.' then $
result = result(1:n_elements(result)-1)
;; Chop off './'
result = STRMID(result,2,1000)
;; Chip out subdirectories (for some reason, the -prune option doesn't
;; work properly, so I have dropped using it).
ix = WHERE(STRPOS(result,'/') EQ -1,count)
IF count EQ 0 THEN RETURN,''
;; Put back the specified (not full) path
result = dir + result(ix)
; End of 'findfile.pro'.