[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: lost data?
> Is there a bug in IDL's Save/Restore command? I've just spent the last 18
> days running a Monte Carlo simulation to seemingly lose all my data. The
> problem occured when our license manager stopped responding (network
> problem) hence the IDL session running the simulation crashed. The MC
> code is designed to save results periodically as it runs (just in case
> this sort of thing happens). I've just tried:
> "restore, 'mc_file.sav', /Verbose"
> only to get:
> % RESTORE: IDL version 5.3 (linux, x86).
> % RESTORE: Truncated save file, restored as much as possible:
> That "resored as much as possible:" is in fact 0 (zero). Despite the file
> itself being 17 Mb! Some of my .sav files are a lot bigger than this, yet
> don't seem to have any problems. Is there anyway to recover this file, or
> prevent this happening again in the future? I'm going to very upset to
> lose 18 days work...
I ran into the same problem a long time ago when I used to run long
FORTRAN jobs on a VAX/VMS system. My batch job wrote data to a netCDF
file as it ran, but if the batch job died, the file would be incomplete
and therefore unusable.
Fortunately, netCDF offers a mechanism that allows you to synchronize an
open file to disk. In IDL, it's done as follows:
ncdf_control, cdfid, /sync
where cdfid is the identifier of an open netCDF file. You will need to
create a netCDF file which has an unlimited dimension; see the example
for ncdf_vardef in the online help for an example.
Another option, which I have not tested, is to write the results from
your simulation to a binary output file (not a SAVE file), and then
periodically execute the flush procedure, e.g.,
where lun is the logical unit number of the file to be synchronized to