File I/O

fluxEngine supports loading measurements from disk and saving them to disk. At the moment the following formats are supported:

  • ENVI for HSI cubes (see below for format-specific notes)

More formats will be added to fluxEngine in the future with the same API that is currently available.

fluxEngine records loaded data in so-called measurement lists. When loading a file from disk all measurements found in the file will be loaded into RAM and stored in a measurement list structure. A measurement list is an array of measurements. Many formats (such as ENVI) can only support one measurement per file, in which case loading such as file will always yield exactly a single measuremet.

Once loaded the data from a measurement can be accessed by the user, or used as input for a model for processing.

To save data fluxEngine currently provides the means to create a measurement list containing a single HSI cube measurement. The data may either be provided by the user directly in form of a tensor, or it may be collected from buffer containers that were filled during data acquistiion from a connected instrument device.

ENVI Format

The ENVI format has established itself as the de-facto standard format for HSI data cubes. It was developed by Harris Geospatial.

It consists of two files: a so-called header file (the extension typically being .hdr) that contains an ASCII description of the data stored in the file, as well as a data file (with varying extensions, depending on the software that writes the data, but .bin or .raw are commonly seen ones) that contains the actual image data in binary form.

Note that the data file will have one of the following formats:

  • By default (file type = ENVI Standard) the binary data is stored as raw bytes directly in the data file.

  • The ENVI Software Suite itself supports loading the actual data in a variety of different formats (such as image formats) instead of just raw data. See the list of ENVI file types for details. This potentially has the advantage of supporting compression, depending on the actual format.

fluxEngine currently only supports the standard binary format with the data being in memory, but does not support the various other defined format variants. However, as the standard binary format is the de-facto standard for hyperspectral data cubes, the vast majority of ENVI files can be loaded by fluxEngine.

fluxEngine does support reading data in all byte orders (both big and little endian), all scalar types defined in the header specification, as well as all storage orders: BIP, BIL and BIP.

fluxEngine will happily load an ENVI file that contains geospatial information, such as coordinate systems, but will ignore it completely. Saving the same measurement again will not include that inforamtion.