File I/O

fluxEngine provides the functionality to load or save recordings from disk. At the moment this is restricted to the ENVI format for HSI cubes.

Loading ENVI cubes

To load an ENVI cube from disk, one may use the loadMeasurementList() method. The user is required to specify the format, which must be "ENVI" in this case (the format is case sensitive!), and the file name.

For example, assuming the cube has been stored in a file cube.hdr (accompanied by its corresponding .bin file), the following code would load that file:

ml = fluxEngine.loadMeasurementList(h, 'ENVI', inputCubeFile)

The result of this method is a measurement list. In the case of ENVI this will always consist of exactly one measurement, but other formats that may be supported in the future might contain more than one measurement per file. The count() method will return the number of measurements in a measurement list, which in this case will be 1.

The following code shows how to extract information about the measurement:

valueType = ml.valueType(0)
wavelengths = ml.wavelengths(0)
data = ml.data(0)
if ml.hasReference(0, "WhiteReference"):
    whiteData = ml.referenceData(0, "WhiteReference")
else:
    whiteData = None
if ml.hasReference(0, "DarkReference"):
    darkData = ml.referenceData(0, "DarkReference")
else
    darkData = None

Note that the white and dark references may not be present. The ENVI parser of fluxEngine will look for files thast have the name cube_White.hdr and WHITEREF_cube.hdr to find the white reference cube in the same directory as the specified file, and cube_Dark.hdr and DARKREF_cube.hdr to find the dark reference cube. If these exist they may be returned by the referenceData() method, otherwise only the cube itself will be loaded.

The information returned here may be used to initialize an offline processing context, see Data Processing for details.

Note

After loading an ENVI cube the storage order of the loaded measurement will always be BIP, regardless of how the cube was stored on disk. This is because fluxEngine internally works in the BIP storage order and automatically converts data when it is loaded by the user.

Creating an ENVI cube from numpy data

To create an ENVI cube the user must provide a three-dimensional numpy array that contains the cube data in any storage order, as well as a single array containing the wavelengths associated with that cube.

This is done via the createMeasurementHSICube() method.

The MeasurementHSICubeInput class exists to allow the user to provide the required data and metadata to create a measurement list object that may be saved. The following example shows how to save data in the form of an ENVI cube, assuming that the data attribute contains the cube data in the specified storage order and the wavelengths attribute contains the associated wavelengths:

input = fluxEngine.MeasurementHSICubeInput()
input.name = 'Test measurement'
input.valueType = fluxEngine.ValueType.Intensity
input.storageOrder = fluxEngine.HSICube_StorageOrder.BSQ
input.wavelengths = wavelengths
input.data = data
input.whiteReference = whiteReferenceData
input.darkReference = None
ml = fluxEngine.createMeasurementHSICube(h, input)

The storage order and wavelengths of the white reference must be the same as that of the data.

To save the measurement in the ENVI format, the following call may be used:

saveReferences = True
fileName = r'C:\...\cube.hdr'
fluxEngine.saveMeasurementList(h, ml, 'ENVI',
                               fileName, saveReferences)

If saveReferences is True the references will be saved next to the ENVI file itself. Otherwise only the data itself will be saved, even if the measurement contains white reference data. The white reference will be stored in the form cube_White.hdr. (The ENVI reader in fluxEngine will also try to look for files of the pattern WHITEREF_cube.hdr for compatibility with other software, but the ENVI writer will only write out white references in the other pattern.)

Note

It is currently not possible to create an ENVI cube in a storage order other than BIP, which is what fluxEngine uses internally. When creating the measurement list via the createMeasurementHSICube() method the storage order is converted internally. The storage order attribute is only to ensure that fluxEngine knows the storage order of the input.

Creating an ENVI cube from camera recordings

Instead of numpy arrays the user may also specify a list of buffer containers of recording contexts to create an ENVI cube of the recorded data.

For this, instead of the MeasurementHSICubeInput class the MeasurementHSICubeBufferInput class should be used instead. It is identical to the other class, but has an attribute bufferContainers that accepts an array of buffer containers instead of the data attribute that accepts a numpy array.

Taking the recording example in the device section, the following code may be used to save the data in an ENVI cube:

recordingInfo = ctx.hsiRecordingResultInfo()
input = fluxEngine.MeasurementHSICubeBufferInput()
input.name = 'Test measurement'
input.valueType = fluxEngine.ValueType.Intensity
input.wavelengths = recordingInfo.wavelengths
input.whiteReference = recordingInfo.whiteReference
input.darkReference = recordingInfo.darkReference
input.bufferContainers = [recordingData]
ml = fluxEngine.createMeasurementHSICube(h, input)
fluxEngine.saveMeasurementList(h, ml, 'ENVI',
                               fileName, True)

To record larger amounts of data, without having to pre-allocate a very large buffer container, the following recording logic might be useful (assuming the user has provided a continueRecording() function, as well as ther other input variables):

# Assume ctx has already been created
recordingData = []
container = fluxEngine.createBufferContainerForRecordingContext(ctx, 500)
recordingData.append(container)
p = fluxEngine.InstrumentDevice.AcquisitionParameters()
# Use a larger amount of buffers for recording
p.bufferCount = 50
camera.startAcquisition(p)
while continueRecording():
    buffer = camera.retrieveBuffer(1000)
    if buffer is not None:
        ctx.setSourceData(buffer)
        ctx.processNext()
        if recordingData[-1].count() == 500:
            # allocate a new buffer container if the old one
            # is full
            container = fluxEngine.createBufferContainerForRecordingContext(ctx, 500)
            recordingData.append(container)
        recordingData[-1].addLastResult(ctx)
        camera.returnBuffer(buffer)
camera.stopAcquisition()
recordingInfo = ctx.hsiRecordingResultInfo()
input = fluxEngine.MeasurementHSICubeBufferInput()
input.name = 'Test measurement'
input.valueType = fluxEngine.ValueType.Intensity
input.wavelengths = recordingInfo.wavelengths
input.whiteReference = recordingInfo.whiteReference
input.darkReference = recordingInfo.darkReference
input.bufferContainers = recordingData
ml = fluxEngine.createMeasurementHSICube(h, input)
fluxEngine.saveMeasurementList(h, ml, 'ENVI',
                               outputFileName, True)