Importing Cubes and Spectra

Using the Import Data menu entry in the File menu the user may import HSI cubes and spectra.

ENVI Cubes

fluxTrainer supports loading cubes stored in the ENVI format. A HSI cube consists of two files: the header file (file extension .hdr) that describes the structure of the cube and also contains metadata for it, and the actual binary data of the cube, stored in a file with the same name as the header file, but a different extension; typically .bin or .dat are used, though others also exist. fluxTrainer will look for a large variety of file extensions to locate the associated binary data for a given header file.

If references are present, fluxTrainer will also look for other ENVI files in the same directory that follow a specific pattern. If a file is called Cube.hdr, then fluxTrainer will automatically load other cubes as references if they match the following patterns (once a reference is found, that reference will not be looked for further; if no reference is found for the cube, the cube is assumed not to have that type of reference):

  • Cube_White.hdr for the white reference cube

  • WHITEREF_Cube.hdr for the white reference cube

  • Cube_Dark.hdr for the dark reference cube

  • DARKREF_Cube.hdr for the dark reference cube

  • Cube_Illumination.hdr for the illumination reference cube

The ENVI standard does not officially specify a way to store the value type of a cube, i.e. whether the cube was stored in intensities, reflectances, absorbances, etc. Cubes exported from fluxTrainer will include a special comment line that includes additional metadata not supported by the ENVI format. Cubes saved by other software will typically not include this information. fluxTrainer implements support for parsing some non-standard extensions by some other recording software to achieve this. Additionally, if the value type of the cube cannot be otherwise determined, if a white reference is present next to a cube, and the white reference is appropriate for the cube, it will be assumed that the cube is stored in intensities.

Wavelength Range Mismatch

If the wavelength range of the cube does not overlap sufficiently with the current wavelength range of the Source (for example when loading a cube measured with an InGaAs camera in a project using the Hyperspectral VIS present), the user will be asked whether to automatically switch the wavelength range of the source while loading the cube:

_images/data_viewer_adjust_wavelength_range.png

This allows the user to update the wavelength range of the Source to better fit the the cube that is being loaded. (The algorithm by which the source wavelength grid is adjusted is the same as when the Loaded Cube button is pressed when editing the parameters of the source.

New Cube Settings

If the value type of the cube could not be determined, the user will be prompted to specify it. For this the following window will open:

_images/data_viewer_new_cube_preview_1.png

At the very top of this window there is a short message that indicates to the user why this window was shown. In this case it is because the value type of the cube could not be detected automatically.

Below that the user has the option of selecting the value type:

_images/data_viewer_new_cube_preview_2.png

Below that the user has the possibility to explicitly load other cubes as potential reference cubes for the given cube. (This is only enabled if the user selected that the cube is stored in intensities or radiances.) Only compatible cubes can be loaded.

Below that there is the Project Settings group box that allow the user to indicate they want to update the parameters of the Source in the project. For example, if the source is currently set to Reflectances, but the user wants to load a cube that is only present in Intensities without a white reference, they can select Work with intensity data here to indicate they want to change the parameters of the Source while loading this cube. (The option may be disabled if other reflectance-only cubes have already been loaded in the project.) Additionally, when indicating to work in intensities, the user can decide to enable or disable the illumination correction. If enabled all loaded cubes must include metadata that they are already illumination corrected, or have an illumination reference.

Finally, at the bottom of the window there is a preview of the loaded cube, both as an image view, as well as a chart view. The views are the same type of views that the user can add to a project themselves.

There is also a button Use selection as white reference, which will extract the average of the current selection and use it as a white reference cube. This is useful when importing a cube where a white reference material was part of the measured image (for example when importing data from a HSI imager camera). Since such a white reference will be averaged over all pixels, it will only provide a spectral reference, and not compensate any inhomogeneitis in the illumination.

The window can only be closed with Ok if the cube can be used in the project. This means:

  • If the project is set to work in reflectances, and the cube is provided in intensities, a white reference cube must be present.

  • If the project is set to work in intensities, but illumination correction is active, an illumination reference cube must be present.

Loading cubes with missing references

A variant of this window will also be shown if the value type of the cube can be determined, but a reference cube could not be found next to the cube that was loaded. This allows the user to manually load a reference cube:

_images/data_viewer_new_cube_preview_3.png

This may be useful in the case where the cube was written by a software that doesn’t follow the supported naming conventions for reference cubes that fluxTrainer supports.

The value type of the cube cannot be changed in this case, as the cube already contained metadata to indicate which value type it has.

The same happens if illumination correction is enabled in the project, but the cube has no illumination reference.

JCAMP-DX Spectra

fluxTrainer also supports the JCAMP-DX format for loading individual spectra. A JCAMP-DX file may contain one or more spectra. JCAMP-DX files may have the file extension .jdx or .dx. When using the Import Data menu entry the user may select a JCAMP-DX file to import.

Since fluxTrainer works with HSI cubes and not individual spectra, a JCAMP-DX file will be transformed into a HSI cube.

File containing a single spectrum

If the JCAMP-DX file contains only a single spectrum, the spectrum will be loaded as if it were a HSI cube with width 1 and height 1.

File containing multiple spectra

If the JCAMP-DX file contains multiple spectra, fluxTrainer will check if the wavelengths and value type of all spectra in the file are identical. If that is not the case, fluxTrainer will be unable to load that file. Otherwise it will combine all spectra into a HSI cube of the following form:

  • The cube will always be the smallest square that can fit the number of spectra in the ENVI file. A file with 2, 3, or 4 spectra will be organized in a cube with width and height 2, a file with 6 spectra will be organized in a cube with width and height 3, etc.

  • The first spectrum will be assigned to the top-left-most pixel of the cube, the second spectrum to the pixel to the right of that. Once the end of a line is reached, the next spectrum will be assigned to the left-most pixel of the next lower line. For example, a cube of width and height 3 will organize the up to 9 spectra of the JCAMP-DX file in the following manner:

    1

    2

    3

    4

    5

    6

    7

    8

    9

  • If there are any remaining pixels in the cube for which there aren’t any spectra in the file, those will be filled with NaN values.