File I/O for Measurements (HSI Cubes, etc.)

MeasurementList

class MeasurementList

A measurement list.

This contains a list of measurements.

Currently objects of this type can only be created by loading data from disk.

Public Functions

MeasurementList() = default

Default constructor.

This creates an empty wrapper object.

inline MeasurementList(MeasurementList &&other)

Move constructor.

Parameters:

other – The measurement list to move

inline MeasurementList &operator=(MeasurementList &&other)

Move assignment operator.

Parameters:

other – The measurement list to move

Returns:

A reference to this

inline ~MeasurementList()

Destructor.

inline int count() const

Get the number of measurements in a measurement list.

Returns:

The number of measurements

inline bool getValueType(int index, ValueType &valueType) const

Get the value type of a measurement.

Get the value type of the data in a given measurement. This indicates a measurement was made in intensities or reflectances.

There is also an overload of this method that is enabled on C++17 compilers that returns std::optional<ValueType> instead, MeasurementList::valueType().

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • valueType – [out] If the value type of the measurement could be determined, and is listed in the ValueType enumeration, it will be stored here, and true will be returned. Otherwise the parameter will not be changed and false will be returned instead.

Returns:

true if a supported value type could be determined for the measurement, false otherwise

inline std::optional<ValueType> valueType(int index) const

Get the value type of a measurement.

Get the value type of the data in a given measurement. This indicates a measurement was made in intensities or reflectances.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

The value type of the measurement, if it is one of the supported types in the ValueType enumeration, or std::nullopt if the value type could not be determined or it is a non-supported type.

inline bool getName(int index, std::string &name)

Get the name of the measurement.

Many file formats allow the user to specify a name for a given measurement. If such a name is present it will be returned.

There is also an overload of this method that is enabled on C++17 compilers that returns std::optional<std::string> instead, MeasurementList::name().

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • name – [out] The name of the measurement will be stored in this variable if a name was set in the measurement

Returns:

true if a name was set in the measurement, false otherwise

inline bool getDescription(int index, std::string &description)

Get the description of the measurement.

Many file formats allow the user to specify a description for a given measurement. If such a name is present it will be returned.

There is also an overload of this method that is enabled on C++17 compilers that returns std::optional<std::string> instead, MeasurementList::description().

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • description – [out] The description of the measurement will be stored in this variable if a description was set in the measurement

Returns:

true if a description was set in the measurement, false otherwise

inline std::optional<std::string> name(int index)

Get the name of the measurement.

Many file formats allow the user to specify a name for a given measurement. If such a name is present it will be returned.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

The name of the measurement, if set, std::nullopt otherwise

inline std::optional<std::string> description(int index)

Get the description of the measurement.

Many file formats allow the user to specify a description for a given measurement. If such a description is present it will be returned.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

The description of the measurement, if set, std::nullopt otherwise

inline Structure structure(int index) const

Get the tensor structure of a measurement.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

The tensor structure of a measurement

inline std::pair<void const*, size_t> rawData(int index) const

Get the raw data of the measurement.

Retrieves a pointer to the first element of the tensor that contains the data of a given measurement.

This will not copy any data, but only return a pointer to the internal data. That pointer will only be valid as long as this measurement list is not freed.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

A pair containing the pointer to the first element of the tensor data of the measurement, and the size of the data in bytes.

inline TensorData tensorData(int index) const

Get a tensor view of the measurement.

Retrieves a tensor view of the data stored in the measurement.

This will not copy any data, but only return a pointer to the internal data. That pointer will only be valid as long as this measurement list is not freed.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

A tensor view of the data in the stored measurement.

inline void copyData(int index, void *data, std::array<int64_t, 5> strides = {}) const

Copy the data of the measurement into a user-supplied buffer.

Create a copy of the data of the measurement. The data will be stored in a buffer provided by the user that has the stride structure specified by the strides parameter.

The user must ensure the buffer is large enough to hold the data. The structure may be obtained via the MeasurementList::structure() method.

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • data – A pointer to the user-supplied buffer where the data will be stored in

  • strides – The stride structure of the user-supplied buffer. Only the elements related to the tensor order will be used

inline CalibrationInfo calibrationInfo(int index) const

Get the calibration info of the measurement.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

The calibration info for the given measurement

inline bool isIlluminationCorrected(int index) const

Get whether the measurement was already illumination corrected.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

Whether the measurement as already illumination corrected

inline bool hasReference(int index, std::string referenceName) const

Does a given measurement have a specific reference?

Sometimes the file format loaders are able to automatically load reference data that was stored in conjunction with the actual measurement. This method returns whether such reference data for a given measurement exists.

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • referenceName – The name of the reference. This must be "WhiteReference", "DarkReference", or "IlluminationReference" at the moment. (This is case-sensitive.)

Returns:

Whether a specific reference measurement is present in the measurement

inline Structure referenceStructure(int index, std::string referenceName) const

Get the tensor structure of a reference measurement.

Sometimes the file format loaders are able to automatically load reference data that was stored in conjunction with the actual measurement. This method returns the tensor structure of such a reference measurement, if present.

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • referenceName – The name of the reference. This must be "WhiteReference", "DarkReference", or "IlluminationReference" at the moment. (This is case-sensitive.)

Returns:

The tensor structure of a measurement

inline std::pair<void const*, size_t> rawReferenceData(int index, std::string referenceName) const

Get the raw data of a reference measurement.

Sometimes the file format loaders are able to automatically load reference data that was stored in conjunction with the actual measurement. This method returns a pointer to the data of such a reference measurement, if present.

This will not copy any data, but only return a pointer to the internal data. That pointer will only be valid as long as this measurement list is not freed.

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • referenceName – The name of the reference. This must be "WhiteReference", "DarkReference", or "IlluminationReference" at the moment. (This is case-sensitive.)

Returns:

A pair containing the pointer to the first element of the tensor data of the reference measurement, and the size of the data in bytes.

inline TensorData referenceTensorData(int index, std::string referenceName) const

Get a tensor view of the data of a reference measurement.

Sometimes the file format loaders are able to automatically load reference data that was stored in conjunction with the actual measurement. This method returns a tensor view of the data of that reference measurement, if present.

This will not copy any data, but only return a pointer to the internal data. That pointer will only be valid as long as this measurement list is not freed.

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • referenceName – The name of the reference. This must be "WhiteReference", "DarkReference", or "IlluminationReference" at the moment. (This is case-sensitive.)

Returns:

A tensor view of the reference measurement data.

inline void copyReferenceData(int index, std::string referenceName, void *data, std::array<int64_t, 5> strides = {}) const

Copy the data of a reference measurement into a user-supplied buffer.

Sometimes the file format loaders are able to automatically load reference data that was stored in conjunction with the actual measurement. This method copies the data of such a reference measurement, if present.

Create a copy of the data of a reference measurement. The data will be stored in a buffer provided by the user that has the stride structure specified by the strides parameter.

The user must ensure the buffer is large enough to hold the data. The structure may be obtained via the MeasurementList::referenceStructure() method.

Parameters:

  • index – The index of the measurements, starting at 0 for the first measurement in the list

  • referenceName – The name of the reference. This must be "WhiteReference", "DarkReference", or "IlluminationReference" at the moment. (This is case-sensitive.)

  • data – A pointer to the user-supplied buffer where the data will be stored in

  • strides – The stride structure of the user-supplied buffer. Only the elements related to the tensor order will be used

inline bool isHsiCube(int index) const

Is a given measurement a HSI cube?

While the only format supported as of now is ENVI, future versions of fluxEngine will support other types of measurements. For this reason this function exists to check whether a measurement that was loaded from disk is actually a HSI cube.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

Whether a specific measurement is a HSI cube

inline std::vector<double> wavelengths(int index) const

Obtain the wavelengths of a given measurement.

Formats that store spectral data, such as ENVI for hyperspectral data cubes, come with information about which wavelengths are assigned to the spectral dimension of the data.

Parameters:

index – The index of the measurements, starting at 0 for the first measurement in the list

Returns:

The list of wavelengths associated with the spectral dimension of the measurement

struct Structure

Measurement structure.

This describes the tensor structure of a measurement. It is returned by either MeasurementList::structure() or MeasurementList::referenceStructure() and describes the tensor that contains the measurement data.

Public Members

DataType dataType = {}

The data type of the measurement.

int order = {}

The tensor order of the measurement.

For HSI cubes this will be 3.

std::array<int64_t, 5> dimensions = {}

The dimensions of the measurement.

Only the first MeasurementList::Structure::order entries will contain valid data.

std::array<int64_t, 5> strides = {}

The strides of the measurement.

Only the first MeasurementList::Structure::order entries will contain valid data.

The strides are only relevant when accessing the internal data directly without making a copy, via the MeasurementList::rawData() or MeasurementList::rawReferenceData() methods.

loadMeasurementList

inline MeasurementList fluxEngine::loadMeasurementList(Handle &handle, std::string format, std::string fileName)

Load a measurement from disk.

This function may be used to load a measurement from disk. At the moment the following formats are supported:

  • Hyperspectral cubes in ENVI format. (Specify "ENVI" as the format; note that this is case-sensitive.)

Please refer to the main fluxEngine documentation for details on the supported file formats.

If the data could be loaded from disk successfully, a MeasurementList object will be returned. Many file formats (for example ENVI) only support storing a single measurement in a file, which means that the list of measurements will always contain exactly one entry for that type of format. Other formats may support storing multiple measurements in a file, in which case number of measurements in the resulting list may vary.

The user must ensure they have enough RAM to load the corresponding measurement. Especially HSI cubes may be quite large and take up quite a bit of RAM.

HSI cubes: when loading HSI cubes their storage order is normalized at load time. All cubes, regardless of their storage order on disk, will be available in BIP storage order. This means that the first dimension of the loaded data will correspond to the y dimension, the second to the x dimension and the third to the wavelength dimension.

Note to Windows users: the file path specified here must be encoded in the local codepage, which is not able to encode all possible file names that Windows supports. It is highly recommended to use the std::wstring overload of this function on Windows, which accepts a wide (“Unicode”) file name and does support all possible file names that Windows supports. If this 8bit version is used on Windows with an encoding different from the local codepage, this method will very likely fail.

Note to non-Windows users: the encoding of the file name is highly dependent on the environment, and may or may not be UTF-8. It is up to the user to specify the file correctly; fluxEngine will pass it directly to the corresponding operating system functions.

Parameters:

  • handle – The fluxEngine handle

  • format – The format of the file to load. Note that the string is case-sensitive.

  • fileName – The file name to load

Returns:

The loaded measurement list

inline MeasurementList fluxEngine::loadMeasurementList(Handle &handle, std::string format, std::wstring fileName)

Load a measurement from disk.

This function is identical to the std::string overload, other than it takes a wide (“Unicode”) filename on Windows systems, to support opening files that can’t be encoded in the local codepage.

Please refer to the documentation of the std::string overload of loadMeasurementList() for details on the behavior of this function beyond the encoding of the filename.

Note

This function is only available on Windows and does not exist on other operating systems.

Parameters:

  • handle – The fluxEngine handle

  • format – The format of the file to load. Note that the string is case-sensitive.

  • fileName – The file name to load

Returns:

The loaded measurement list

saveMeasurementList

inline void fluxEngine::saveMeasurementList(Handle &handle, MeasurementList const &list, std::string format, std::string fileName, bool saveReferences)

Save a measurement (list) to disk.

This function may be used to save a measurement (list) to disk. At the moment the following formats are supported:

  • Hyperspectral cubes in ENVI format. (Specify "ENVI" as the format; note that this is case-sensitive.)

Please refer to the main fluxEngine documentation for details on the supported file formats.

The measurements in the supplied measurement list must be compatible with the save format, otherwise an error will occur. In particular, some formats, such as ENVI, only support saving a single measurement in a file.

HSI cubes: note that this routine always stores HSI cubes in the BIP storage order when the format allows for a choice in that.

Licensing: this function may be called even if the device a license has been tied to has since been disconnected, or the license dongle has since been removed, in order to allow the application gracefully save data before failing. However, if the license is tied to a device, the device must have been connected at least once with the current handle before this function can be called successfully.

Note to Windows users: the file path specified here must be encoded in the local codepage, which is not able to encode all possible file names that Windows supports. It is highly recommended to use the std::wstring overload of this function on Windows, which accepts a wide (“Unicode”) file name and does support all possible file names that Windows supports. If this 8bit version is used on Windows with an encoding different from the local codepage, this method will very likely fail.

Note to non-Windows users: the encoding of the file name is highly dependent on the environment, and may or may not be UTF-8. It is up to the user to specify the file correctly; fluxEngine will pass it directly to the corresponding operating system functions.

Parameters:

  • handle – The fluxEngine handle

  • list – The list of measurements to save

  • format – The format of the file to load. Note that the string is case-sensitive.

  • fileName – The file name to load

  • saveReferences – Whether to also save reference measurements (if present), or only save the actual measurement itself.

inline void fluxEngine::saveMeasurementList(Handle &handle, MeasurementList const &list, std::string format, std::wstring fileName, bool saveReferences)

Save a measurement (list) to disk.

This function is identical to the std::string overload, other than it takes a wide (“Unicode”) filename on Windows systems, to support opening files that can’t be encoded in the local codepage.

Please refer to the documentation of the std::string overload of saveMeasurementList() for details on the behavior of this function beyond the encoding of the filename.

Note

This function is only available on Windows and does not exist on other operating systems.

Parameters:

  • handle – The fluxEngine handle

  • list – The list of measurements to save

  • format – The format of the file to load. Note that the string is case-sensitive.

  • fileName – The file name to load

  • saveReferences – Whether to also save reference measurements (if present), or only save the actual measurement itself.

MeasurementHSICubeInput

struct MeasurementHSICubeInput

Input data for creating a HSI cube measurement.

This structure describes the information required to create a HSI cube measurement from user-supplied data.

Public Members

std::string name = {}

The name of the measurement.

If an empty string this indicates the measurement has no specific name.

DataType dataType = {DataType::UInt8}

The data type of the data.

ValueType valueType = {ValueType::Intensity}

The value type of the measurement.

This indicates if the measurement was made in intensities or reflectances.

HSICube_StorageOrder storageOrder = {HSICube_StorageOrder::BIP}

The storage order of the user-supplied data.

Note that internally the storage order will always be converted to BIP storage order. This here indicates only the order in which the user has stored the data.

int64_t lines = {}

The number of lines in the HSI cube.

In BIP and BIL storage order this will correspond to the outer (left-most) dimension of the cube, in BSQ storage order to the middle dimension of the cube.

int64_t samples = {}

The number of samples in the HSI cube.

In BIP storage order this will correspond to the middle dimension of the cube, in BIL and BSQ storage order to the inner (right-most) dimension of the cube.

int64_t bands = {}

The number of bands in the HSI cube.

In BIP storage order this will correspond to the inner (right-most) dimension of the cube, in BIL storage order to the middle dimension of the cube, and in BSQ storage order to the outer (left-most) dimension of the cube.

std::array<int64_t, 3> strides = {}

The strides of the cube.

These must be the strides ordered according to the actual memory layout.

For example, if the data resides contiguously in memory (without any holes) and hence has a trivial stride structure, and a cube with 512 lines, 320 samples and 256 bands is to be saved, the strides should be specified as follows:

  • BIP storage order: dimensions are {512, 320, 256}, hence strides should be {81920, 256, 1}

  • BIL storage order: dimensions are {512, 256, 320}, hence strides should be {81920, 320, 1}

  • BSQ storage order: dimensions are {256, 512, 320}, hence strides should be {163840, 320, 1}

void const *data = {}

A pointer to the first element of the cube data.

std::vector<double> wavelengths

A vector of the wavelengths of the cube.

The number of wavelengths are specified by the bands member.

MeasurementHSICubeInput const *whiteReference = {}

Optional: the white reference measurement.

If not nullptr this should point to a structure of the same type that contains the data of the white reference.

MeasurementHSICubeInput const *darkReference = {}

Optional: the dark reference measurement.

If not nullptr this should point to a structure of the same type that contains the data of the dark reference.

MeasurementHSICubeInput const *illuminationReference = {}

Optional: the illumination reference measurement.

If not nullptr this should point to a structure of the same type that contains the data of the illumination reference.

CalibrationInfo const *calibrationInfo = {}

Optional: calibration info.

If not nullptr this should point to an instance of a CalibrationInfo object that should be used for the calibration information of the measurement that is to be stored.

bool isIlluminationCorrected = {}

Optional: indicate that the measurement has already been illumination corrected.

If the measurement is in intensities (or radiances), if this is true it indicates that the illumination correction has already been performed for this measurement.

MeasurementHSICubeBufferInput

struct MeasurementHSICubeBufferInput

Input data for creating a HSI cube measurement from buffer containers.

This structure describes the information required to create a HSI cube measurement a list of BufferContainer objects that the user has used to record data via a recording processing context.

Public Members

std::string name

The name of the measurement.

If an empty string this indicates the measurement has no specific name.

ValueType valueType = {ValueType::Intensity}

The value type of the measurement.

This indicates if the measurement was made in intensities or reflectances.

std::vector<BufferContainer const*> bufferContainers

Pointers to the buffer containers to concatenate.

The structure of the buffer containers is verified to be compatible with a HSI cube structure. It is assumed that the data in the containers is in BIP storage order, as that is the storage order returned by all recording processing contexts.

All buffer containers must match in structure, but may contain a varying number of lines. They will all be concatenated.

std::vector<double> wavelengths

The vector of wavelengths of the measurement.

Its size will be cross-checked against the structure of the buffer containers supplied.

ProcessingContext::ReferenceMeasurement whiteReference

Optional: the white reference.

The user may provide the same structure that is also returned by the context creation functions to indicate that the white reference should be saved together with the measurement.

ProcessingContext::ReferenceMeasurement darkReference

Optional: the dark reference.

The user may provide the same structure that is also returned by the context creation functions to indicate that the dark reference should be saved together with the measurement.

ProcessingContext::ReferenceMeasurement illuminationReference

Optional: the illumination reference.

The user may provide the same structure that is also returned by the context creation functions to indicate that the illumination reference should be saved together with the measurement.

CalibrationInfo const *calibrationInfo = {}

Optional: calibration info.

If not nullptr this should point to an instance of a CalibrationInfo object that should be used for the calibration information of the measurement that is to be stored.

bool isIlluminationCorrected = {}

Optional: indicate that the measurement has already been illumination corrected.

If the measurement is in intensities (or radiances), if this is true it indicates that the illumination correction has already been performed for this measurement.

createMeasurementHSICube

inline MeasurementList fluxEngine::createMeasurementHSICube(Handle &handle, MeasurementHSICubeInput const &input)

Create a new HSI cube measurement from raw data.

Creates a measurement list that contains a single HSI cube measurement from raw data that the user must supply. Please see the MeasurementHSICubeInput structure for further details on the data that must be provided for this to succeed.

The resulting measurement list may then be saved via one of the overloads of the saveMeasurementList() functions.

A copy of the data provided by the user will be created in this function. The user should ensure that there is enough RAM available to store such a copy.

Parameters:

  • handle – The fluxEngine handle

  • input – The input parameters required to create the measurement

Returns:

The newly created measurement list

inline MeasurementList fluxEngine::createMeasurementHSICube(Handle &handle, ProcessingQueueSet &processingQueueSet, MeasurementHSICubeInput const &input)

Create a new HSI cube measurement from raw data.

Creates a measurement list that contains a single HSI cube measurement from raw data that the user must supply. Please see the MeasurementHSICubeInput structure for further details on the data that must be provided for this to succeed.

The resulting measurement list may then be saved via one of the overloads of the saveMeasurementList() functions.

A copy of the data provided by the user will be created in this function. The user should ensure that there is enough RAM available to store such a copy.

This overload allows the user to specify a processing queue set to perform any data processing that may be required when creating the cube. If an invalid processing queue set is specified the default processing queue set of the handle will be used, instead, which is equivalent to calling the overload without the processingQueueSet parameter.

Parameters:

  • handle – The fluxEngine handle

  • processingQueueSet – The processing queue set to use for processing

  • input – The input parameters required to create the measurement

Returns:

The newly created measurement list

inline MeasurementList fluxEngine::createMeasurementHSICube(Handle &handle, MeasurementHSICubeBufferInput const &input)

Create a new HSI cube measurement from PushBroom buffers.

This will create a new HSI cube measurement from a set of PushBroom buffer containers the user must provide. This is useful if the user created one or more buffer containers via the createBufferContainer() overload that accepts a recording context, and added data to them via BufferContainer::addLastResult() to record data from a device. In that case the recording can be combined to a HSI cube that the user may then save to disk.

The list of buffers should be provided via the input parameter in form of a MeasurementHSICubeBufferInput structure. See the documentation of that structure for details on what is required to be specified.

The resulting measurement list may then be saved via one of the overloads of the saveMeasurementList() functions.

A copy of the data provided by the user will be created in this function. The user should ensure that there is enough RAM available to store such a copy.

Parameters:

  • handle – The fluxEngine handle

  • input – The input parameters required to create the measurement

Returns:

The newly created measurement list