Re: Large File Plotting

["J.D. Smith" <jdsmith(at)astro.cornell.edu>]

Mark Przeslawski wrote:
> 
> Does anyone have an example of plotting
> very large time history data files. I tried
> creating a 1 million point file and it plotted
> okay. I tried a 10 million point file with
> various plotting routines and it hung my
> Windows 2000 machine (PIII, 700MHz,
> 256Mb RAM). I tried plot, live_plot,
> and some of the coyote plotting
> routines to no avail. What is needed is
> something that can detect file size and
> only display to the pixel resolution of the
> screen.
> 
> Also, does anyone have experience pushing
> this size data through the signal processing
> routines? These are not atypical file sizes
> that I need to handle.


With this many points, 2-d plots will simply not convey the information
contained in the data, no matter what the display medium.  You could
indeed use averaging or exclusion techniques to reduce the number of
points plotted.  A better technique in many cases is to convert your 2d
value vs. time data into a 3-d value-time-density image, where density
is the number density, i.e. dN/dvdt.  You generate the density by
binning into an appropriate number of 2d time*value elements, including
estimated errors if necessary (individual points smeared out with
varying weight over several bins).  This image can be displayed inside
of the plot region using whatever colormap and scaling suits you, or
even with a contour map.

Good luck,

JD

-- 
 J.D. Smith                  |   WORK: (607) 255-6263
 Cornell Dept. of Astronomy  |         (607) 255-5842
 304 Space Sciences Bldg.    |    FAX: (607) 255-5875  
 Ithaca, NY 14853            |