Sparse disk storage for mainly-zero arrays

Versions of GEMPACK released in 2006 or later allow mainly-zero real arrays to be stored on disk in sparse form. This saves hard disk space (but does not reduce the RAM requirements of running programs).

When a newer program writes a Header Array in sparse form, it just writes the nonzero values in the array and the corresponding positions (integers) to the file. Arrays of integers and character strings are never written in sparse form.

GEMPACK programs created prior to 2006 will fail when attempting to read the new sparse files (with error message "un-recognized storage format 'SPSE'"). Therefore a method is needed to switch off the new sparse storage feature.

In ViewHAR the File Options menu item allows you to control whether sparse arrays are stored on disk in sparse format. The Use sparse disk storage checkbox causes real matrices written by ViewHAR to be stored in sparse form, if they are mostly zeroes. Warning: if you want to retain compatibily with earlier GEMPACK programs, make sure the option is switched OFF. ViewHAR can still read sparse matrices, whether or not this option is enabled. The setting in ViewHAR will also apply to other GEMPACK-related utilities written in Delphi Pascal (eg, ViewSOL, AnalyseGE, DiffHAR or AggHAR).

For GEMPACK Fortran programs such as GEMSIM or TABLO-generated Fortran programs, a CMF file option enables the sparse storage feature [WHS = yes|NO ;].

The option applies only to creating or writing hard disk files. All GEMPACK programs created 2006 or later will read sparse arrays whether or not the the sparse storage feature is enabled.

For GEMPACK versions 9 and 10.0, the default option will be that sparse storage is NOT used. For GEMPACK 10.0-002 (April 2010) and later versions, sparse storage will be used by default by Fortran-based programs. Since September 2009, ViewHAR and other Pascal-based programs use sparse storage by default.

A command line program, SPRS2FULL.EXE, may be used to convert sparse storage HAR files to the non-sparse format -- which older GEMPACK programs can read.

Note: The registry key which controls whether ViewHAR uses sparse storage is:

HKEY_CURRENT_USER\Software\GEMPACK\UseSparse

URL of this topic: www.copsmodels.com/webhelp/viewhar/hc_sparsestorage.htm

Link to full GEMPACK Manual

Link to GEMPACK homepage