Device Connection

Library Setup

As described in Driver Directory Structure drivers are loaded in a separate process and are searched for in a directory the user must specify.

In the .NET API the driver directory may be set via the SetDriverBaseDirectory() method:

1 try
2 {
3     handle.SetDriverBaseDirectory(@"C:\Path\To\drivers");
4 }
5 catch (Exception e)
6 {
7     Console.WriteLine($"Error: {e.Message}");
8 }

Enumerating Devices

Before device connection is possible the user must enumerate devices and drivers. This can be done with the DeviceEnumeration.EnumerateDevices() method:

 1 try
 2 {
 3     LuxFlux.fluxEngineNET.DriverType? type = null;
 4     var timeout = new TimeSpan(0, 0, 5); // 5 seconds
 5     LuxFlux.fluxEngineNET.EnumerationResult enumeratedDevices = LuxFlux.fluxEngineNET.DeviceEnumeration.EnumerateDevices(handle, type, timeout);
 6 }
 7 catch (Exception e)
 8 {
 9     Console.WriteLine($"Error: {e.Message}");
10 }

The user may select the driver type to enumerate. There are currently three options:

The user must also specify a timeout for the enumeration process. The enumeration will take exactly that long and then return all devices and drivers that could be found within that time.

Note

For cameras with a GigE Vision interface a timeout larger than three seconds (recommended: at least four seconds) is required, as the standard specifies that as the timeout for device discovery, and the enumeration process does need to load the driver, so using exactly three seconds is likely not enough.

Error Reporting during Enumeration

If an exception is thrown during the call to DeviceEnumeration.EnumerateDevices() this indicates that the enumeration process failed entirely. If an error occurs with a single driver (for example, because it couldn’t be loaded because a dependent DLL was not found) the enumeration process itself will still succeed, but the driver-specific errors will be reported as part of the enumeration result.

Additionally, some drivers may provide warnings during the enumeration process. For example, if a driver detects that the system configuration is such that it would never find any device, it may issue such a warning.

Accessing these errors and warnings may be done via the EnumerationResult structure. The following code would print out all errors and warnings that were encountered during the enumeration process.

1 foreach (fluxEngineNET.EnumerationError error in result.Errors)
2 {
3     Console.WriteLine("Enumeration error: " + error.Message);
4 }
5 foreach (fluxEngineNET.EnumerationWarning warning in result.Warnings)
6 {
7     Console.WriteLine("Enumeration warning: " + warning.Message);
8 }

Enumerated Drivers and Devices

The enumeration result will also contain all drivers that were enumerated (even if they could not be loaded) as well as the devices that were found.

Each driver that was found has a name, which is given by the name of the directory the driver was found in. For example, the virtual PushBroom driver will have the name "VirtualHyperCamera". That information, stored in EnumeratedDriver.Name, will always be present for every enumerated driver. Additionally a state is provided via the EnumeratedDriver.State member, that indicates whether the driver was loaded successfully, or whether it crashed during enumeration, for example.

The lists of drivers and devices are vectors of unique pointers to allow cross-referencing between them (and cross-referencing the drivers from the warning and error lists).

The following code will list all drivers that were found during that enumeration process:

1 foreach (fluxEngineNET.EnumeratedDriver driver in result.Drivers)
2 {
3     Console.WriteLine($"Found driver {driver.Name} of type {driver.Type} in state {driver.State} with {driver.Devices.Length} devices.");
4 }

Finally, there are the devices that were found. These may be accessed either via the EnumerationResult.Devices member to obtain a list of all devices, or the EnumeratedDriver.Devices member of a specific driver to obtain only the devices associated with that driver.

Each enumerated device is stored in a EnumeratedDevice structure that has the following quantities associated with it:

  • A driver-specific id for this device. This id must be provided in conjunction with the type and name of the driver when attempting to connect to a device.

    This id is not stable across system reboots or even unplugging a device and plugging it back in again. It is only considered stable for a short amount of time after enumerating all devices in order to be useful when conecting to it. The contents may also change depending on the driver version used. Never store this id long-term.

    Typically the user should always enumerate all devices, find the device they want to connect to in the list of devices, and then use that id for the connection process.

  • A display name that contains a human-readable device name that may be used when e.g. showing a list of available devices to the user, or when logging a connection attempt.

  • The manufacturer of the device.

  • The model name of the device.

  • Optionally: a serial number of the device. Some devices do not have serial numbers, in which case this will always be empty for such devices. But other devices may indeed have a serial number, but that number is not accessible during enumeration, perhaps because obtaining it requires connecting to the device. In that case this might also be empty, even if the device has a serial number.

    It is guaranteed though that if this is present this will contain the same string as Device.SerialNumber will contain.

  • A ParameterList structure that describes the connection parameters.

    Some devices may be connected directly and no additional information is required. In that case this may be ignored.

    Other devices may require additional information to allow for a successful connection. This may be calibration data that is required for the driver to work – many hyperspectral cameras do not store the list of wavelengths on-camera and fluxEngine drivers for these cameras require them to be supplied in the form of a calibration file.

    Yet another possibility are “manually connected” devices. While many devices that are supported by fluxEngine can be enumerated (because the use an interface that supports this, such as USB), some devices can’t be. In those cases the user must explicitly specify a port, or an address, or something similar in order to connect to that device. Connection parameters may also be used for this.

    The ParameterList structure provided here allows the user to introspect the expected connection parameters by the driver for that specific device. This structure is not required to be used when connecting to a device, if no connection parameters are required, or the user already knows the format for these parameters.

Note

It is possible that an enumeration process does not yield any devices - for example, if no device is currently connected. This is not considered to be an error by EnumerateDevices().

The following code shows to how find the virtual PushBroom device from the list of enumerated devices:

 1 fluxEngineNET.EnumeratedDevice foundDevice = null;
 2 foreach (fluxEngineNET.EnumeratedDevice device in result.Devices)
 3 {
 4     if (device.Driver && device.Driver.Name == "VirtualHyperCamera")
 5         foundDevice = device;
 6         break;
 7     }
 8 }
 9
10 if (foundDevice == null) {
11     Console.WriteLine("Could not find VirtualHyperCamera driver.");
12     return;
13 }

Connecting to a Device

After the user has selected a device (from the list of enumerated devices) to connect to, they may use the method DeviceGroup.Connect() to perform the connection.

This introduces the concept of a device group: some devices may have subdevices that perform different functions, but are accessed through the same interface. For this reason fluxEngine provides the additional abstraction of the device group – the resulting object of a connection process is a device group that will contain at least one device. All operations are performed on the individual devices, except for registering event notifications, and disconnecting.

In the vast majority of cases the device group will consist of only a single device.

The DeviceGroup.Connect() function expects a data structure that contains the information required to perform the connection, ConnectionSettings.

The fields ConnectionSettings.DriverName and ConnectionSettings.DriverType have to be filled according to the Name and Type fields of the EnumeratedDriver of the device the user wants to connect to.

The field ConnectionSettings.Id must be set to the value obtained in Id for the enumerated device.

The user must also specify a timeout, after which the driver process will forcibly be killed and the connection attempt will be assumed to be unsuccessful. This must be set in ConnectionSettings.Timeout. Note that for many devices a timeout of 2 minutes is recommended, and less than 10 seconds will not be enough time in the vast majority of cases.

If the user wants to specify connection parameters they may do so by inserting them into ConnectionSettings.ConnectionParameters, which is a Dictionary<string, string>. Please refer to the documentation of that member for the proper encoding of the various different values. Of note is that file names on Windows systems must be encoded as UTF-8 and not the local code-page.

The following example shows how to connect to a virtual PushBroom camera that was found in the previous example after enumeration. The virtual PushBroom camera requires at least one connection parameter, "Cube" indicating the ENVI cube to load, but supports two more parameters for the white and dark reference cubes, "WhiteReferenceCube and "DarkReferenceCube.

 1 LuxFlux.fluxEngineNET.DeviceGroup deviceGroup = null;
 2 LuxFlux.fluxEngineNET.Device device = null;
 3 try
 4 {
 5     var settings = new LuxFlux.fluxEngineNET.ConnectionSettings();
 6     settings.DriverName = foundDevice.Driver.Name;
 7     settings.DriverType = foundDevice.Driver.Type;
 8     settings.Id = foundDevice.Id;
 9     settings.Timeout = new TimeSpan(0, 1, 0); // 1 minute
10     settings.PassThroughErrorOutput = true;
11     settings.ConnectionParameters = new Dictionary<string, string>();
12     settings.ConnectionParameters["Cube"] = @"C:\Cube.hdr";
13     settings.ConnectionParameters["WhiteReferenceCube"] = @"C:\Cube_White.hdr";
14     settings.ConnectionParameters["DarkReferenceCube"] = @"C:\Cube_Dark.hdr";
15
16     deviceGroup = LuxFlux.fluxEngineNET.DeviceGroup.Connect(handle, settings);
17     device = deviceGroup.PrimaryDevice;
18 catch (Exception e)
19 {
20     Console.WriteLine($"Error: {e.Message}");
21 }

DeviceGroup.Connect() blocks until the connection attempt has completed or there was an error. If connecting to the device does not succeed, or the connection attempt times out, an exception will be thrown.

Disconnecting from a Device

There are three methods to disconnect from a device:

  • Let the DeviceGroup object be finalized by the .NET Garbage Collector. This will forcibly disconnect the device by killing the driver process. However, since the garbage collector may wait an arbitrary amount of time before finalizing an object, this is not recommended.

  • Disposing of the device group via the Dispose() method. This will forcibly disconnect the device by killing the driver process.

  • Call Disconnect() that allows the user to specify a timeout to use while attempting a disconnect operation. This is the recommended method as it allows the driver to properly free its resources. However, this call will block the caller for up to the specified timeout.

    Important: Calling Disconnect() will also dispose of the object!

Devices of a device group that has been disconnected and/or disposed cannot be accessed anymore. (In the .NET API it is safe to call methods on those objects, but those methods will fail.)

Accessing Device Parameters

Most devices can be controlled via parameters. Introspection for these parameters is available via the Device.ParameterList() method. There are three parameter lists for different purposes:

Parameters may be read via the following methods:

For example, most instrument devices will have the standard ROI parameters, such as "Width" and "OffsetX". These will be integers and may be read in the following manner:

1 try
2 {
3     Int64 offsetX = device.GetParameterInteger("OffsetX");
4     Int64 width = device.GetParameterInteger("Width");
5 }
6 catch (Exception e)
7 {
8     Console.WriteLine($"Error: {e.Message}");
9 }

Parameters may be changed via the following methods:

The following example shows how the exposure time of a camera may be altered, assuming that camera uses a floating point exposure time in milliseconds (this can be queried via introspection, if required):

1 try
2 {
3     device.SetParameterFloat("ExposureTime", 3.5);
4 }
5 catch (Exception e)
6 {
7     Console.WriteLine($"Error: {e.Message}");
8 }

Note

Changing parameters for instrument devices when acquisition is currently active will likely cause them to stop acquisition automatically in order to be able to change these parameters. In that case the status of the device will enter the InstrumentDevice.Status.ForcedStop state. The user must then acknowledge this by calling InstrumentDevice.StopAcquisition() and only then may restart acquistion via InstrumentDevice.StartAcquisition().

Some parameter changes may not require acquisition to be stopped though, in which case the acquisition will continue to be active after such a change. Which parameters will stop acquisition will depend on the specific device. There is also no means of querying this information before making such a change, as stopping acquisition may only occur when a specific value is set, or when the device is in a specific state.

The virtual PushBroom driver only has a single parameter that indicates in what interval (in milliseconds) the individual frames should be provided.

1 try
2 {
3     // Virtaul PushBroom: send a frame every 5ms
4     device.SetParameterInteger("Interval", 5);
5 }
6 catch (Exception e)
7 {
8     Console.WriteLine($"Error: {e.Message}");
9 }

Accessing Instrument Devices

For instrument devices the resulting device will be of type InstrumentDevice. Using an appropriate cast one may obtain the correct object:

 1 try {
 2     LuxFlux.fluxEngineNET.Device device = deviceGroup.PrimaryDevice;
 3     var instrumentDevice = (LuxFlux.fluxEngineNET.InstrumentDevice)device;
 4     if (instrumentDevice == null)
 5         throw Exception("The primary device is not an instrument (not expected).");
 6 }
 7 catch (Exception e)
 8 {
 9     Console.WriteLine($"Error: {e.Message}");
10 }

Instrument Device Setup

Directly after having connected to the instrument device the user must set up the shared memory region. The shared memory logic is described in further detail in the Instrument Buffers and Shared Memory section.

To setup the shared memory area one may use the method InstrumentDevice.SetupInternalBuffers(). This method may only be called once for the instrument device and the setting that has been chosen here will be used as long as the device is connected.

As the number of buffers actually used for specific acquisition phases can be specified at a later point in time, but must at most be the number specified here, it is recommended to err on the higher side for this setting, and potentially reduce the number of buffers when starting acquisition.

The trade-off for the number of buffers actually used is the following: more buffers reduce the chance of a dropped frame, but use more RAM and may dramatically increase the latency, i.e. the time it takes between the instrument measuring data and that data being processed by fluxEngine.

For inline data processing a number of 5 (the mimimum) is recommended to avoid issues with high latencies; when recording data higher numbers may be more useful, such as 100. Using more than 100 buffers is not recommended in most cases. (Though only the amount of RAM and possibly quirks of the operating system’s shared memory logic will limit the maximum number specified here.)

 1 try
 2 {
 3     // Use 16 as a compromise value here for now,
 4     // but for low-latency acquisition we may choose
 5     // to only use 5 of these buffers.
 6     instrumentDevice.SetupInternalBuffers(16);
 7 }
 8 catch (Exception e)
 9 {
10     Console.WriteLine($"Error: {e.Message}");
11 }

Data Acquisition

To acquire data from an instrument one must first call the InstrumentDevice.StartAcquisition() method. Afterwards the instrument will start providing buffers in the internal queue that may be retrieved via InstrumentDevice.RetrieveBuffer(). After the data from the buffer has been used the user must return the buffer via InstrumentDevice.ReturnBuffer(). Finally, once acquisition is no longer required, the user may stop acquisition via InstrumentDevice.StopAcquisition().

The InstrumentDevice.StartAcquisition() requires the user to specify additional parameters required for the acquisition. These are:

  • The name of the reference to measure. Some instruments have the ability to perform special operations when the user wants to measure a reference.

    For example, some cameras have a shutter that the driver will automatically close when measuring a dark reference. This has the advantage that the user doesn’t need to turn of the light for these types of cameras.

    Also, the virtual PushBroom driver will return different data (from the various different cubes specified during connection) depending on whether a reference measurement was selected.

    Use "WhiteReference" to specify that a white reference is to be measured, and use "DarkReference" to specify that a dark reference is to be measured. Use "" (and empty string, the default) to specify that regular data is to be measured.

  • The actual number of buffers to use for this acquisition process. This allows the user to reduce the number of buffers actually used during this specific acquisition process. The largest number that may be specified here is the number supplied to InstrumentDevice.SetupInternalBuffers(). If 0 (the default) is specified it will use exactly the number of buffers provided to InstrumentDevice.SetupInternalBuffers().

The following example code shows a typical acquisition loop:

 1 try {
 2     var parameters = new LuxFlux.fluxEngineNET.InstrumentDevice.AcquisitionParameters();
 3     // Keep the parameters at their default
 4     instrumentDevice.StartAcquisition(parameters);
 5     // Acquire 10 frames
 6     for (int i = 0; i < 10; ++i) {
 7         LuxFlux.fluxEngineNET.Buffer buffer = instrumentDevice.RetrieveBuffer(new TimeSpan(0, 0, 1));
 8         if (buffer == null)
 9             continue;
10         try {
11             // Do something with buffer here
12         }
13         finally
14         {
15             instrumentDevice.ReturnBuffer(buffer);
16         }
17     }
18     instrumentDevice.StopAcquisition();
19 }
20 catch (Exception e)
21 {
22     Console.WriteLine($"Error: {e.Message}");
23 }

Measuring References

HSI measurements typically require at least a white reference to be useful for most HSI algorithms. This is because most HSI data processing is done with reflectance values.

fluxEngine provides some facilities to make measuring a reference easier. The primary class here is BufferContainer, which allows the user to store multiple buffers it acquired from a device, and then later reuse them when setting up data processing.

In principle a single buffer could be used as a reference measurement, but in practice it is better to measure multiple buffers, so that noise can be reduced. For many measurements using 10 buffers is a good number, but for very percise measurements 100 buffers or more may be required. (Using more buffers will obviously also take longer and use more RAM.)

The following example code shows how a white and dark reference may be measured with fluxEngine. They will be stored in a buffer container each, which then later may be passed to fluxEngine when setting up data processing.

 1 LuxFlux.fluxEngineNET.BufferContainer whiteReference = null, darkReference = null;
 2 var timeout_1s = new TimeSpan(0, 0, 1);
 3 try
 4 {
 5     // Average 10 frames for the white reference
 6     whiteReference = LuxFlux.fluxEngineNET.Util.CreateBufferContainer(instrumentDevice, 10);
 7     var parameters = new LuxFlux.fluxEngineNET.InstrumentDevice.AcquisitionParameters();
 8     parameters.ReferenceName = "WhiteReference";
 9     // insert code here to prompt user to insert the white
10     // reference underneath the sensor
11     instrumentDevice.StartAcquisition(parameters);
12     while (whiteReference.Count < 10)
13     {
14         LuxFlux.fluxEngineNET.Buffer buffer = instrumentDevice.RetrieveBuffer(timeout_1s);
15         if (buffer == null)
16             continue;
17         whiteReference.Add(buffer);
18         instrumentDevice.ReturnBuffer(buffer);
19     }
20     instrumentDevice.StopAcquisition();
21
22     // Average 10 frames for the dark reference
23     darkReference = LuxFlux.fluxEngineNET.Util.CreateBufferContainer(instrumentDevice, 10);
24     parameters.ReferenceName = "DarkReference";
25     // insert code here to prompt user to close the lid
26     // of the sensor optics
27     instrumentDevice.StartAcquisition(parameters);
28     while (darkReference.Count < 10)
29     {
30         LuxFlux.fluxEngineNET.Buffer buffer = instrumentDevice.RetrieveBuffer(timeout_1s);
31         if (buffer == null)
32             continue;
33         darkReference.A(buffer);
34         instrumentDevice.ReturnBuffer(buffer.id);
35     }
36     instrumentDevice.StopAcquisition();
37
38     // insert code here to prompt the user to open
39     // the lid again after the dark reference has
40     // been measured
41 }
42 catch (Exception e)
43 {
44     Console.WriteLine($"Error: {e.Message}");
45 }

Persistent Buffers

Buffers must be returned to fluxEngine quickly. For this reason there is another data structure available, PersistentBuffer, that allows the user to store data from an individual buffer with a lifetime that solely depends on the object lifetime of the persistent buffer that was allocated.

Persistent buffers may be created in the following manner:

  • A persistent buffer that doesn’t contain any data may be allocated via the InstrumentDevice.AllocatePersistentBuffer() method. It is up to the user to fill that persistent buffer afterwards. (See below for how this may be done.)

    The persistent buffer allocated in this manner will have the same structure as a buffer that would have been returned from the device.

  • A persistent buffer that is a copy of an existing buffer may be created via the Buffer.Copy() method of the Buffer class.

  • A persistent buffer may be duplicated via the PersistentBuffer.Clone() method.

  • A single stored buffer in a buffer container may be extracted as a newly created persistent buffer via the BufferContainer.CopyBuffer() method.

A persistent buffer may be used in any place where a buffer from a device could be used instead. For example, it may be added to a buffer container, it may also be used as input for a processing context (see the Data Processing chapter).

Additionally there is functionality of replacing the contents of an existing persistent buffer with other data (so that constant allocations and deallocations can be avoided):

  • The data of a buffer can be copied into a persistent buffer via the Buffer.CopyInto() method. The structure of both objects must match for this to succeed.

  • The data of an existing persistent buffer can be copied into another persistent buffer via the PersistentBufferInfo.CopyInto() method. The structure of both objects must match for this to succeed.

  • A single stored buffer in a buffer container may be extracted into an existing persistent buffer via the BufferContainer.CopyInto() method.

Using data from an Instrument

Most hyperspectral cameras don’t produce data that can be used as-is, but at least some processing has to happen first. fluxEngine provides the means for this, but these details are described in the Data Processing chapter.

Handling Notifications from a Device

Devices may produce notifications that may be of interest to the user. For example, if the device is unplugged while it is connected, that would trigger a notification with most drivers.

Notifications from devices are kept in a queue in the DeviceGroup object. The user may retrieve the first notification in the queue via DeviceGroup.NextNotification(). That will return either an object of type DeviceGroup.Notification or null if there is currently no notification in the notification queue of the device.

Please refer to the reference documentation of DeviceGroup.Notification. and DeviceGroup.NotificationType for details on the type of notifications.

In order to avoid having to constantly poll for notifications, it is also possible to obtain an operating system handle (HANDLE type on Windows, a file descriptor on all other operating systems) that may be used to include in an event loop. This may be done via the unsafe DeviceGroup.NativeNotificationEventHandle property. (There is currently no safe method of waiting for notifications, this is subject to change in the future.)