Introduction to Hyperspectral Imaging
This introduction describes the key concepts of hyperspectral imaging. It gives an overview over cameras, imaging setups, light, reference measurements and calibration files.
Pushbroom Cameras
The most common type of hyperspectral imaging camera used in industrial applications is the so-called pushbroom line scanner. The most important difference between a regular camera and a pushbroom is that pushbroom cameras only image a single line, not an entire image.
They use a 2D image sensor to image a single line in one dimension of the 2D sensor, diffraction optics are used to map wavelengths onto the other dimension of the 2D sensor.
At the very basic level a slit is used to isolate just a single line of the image, which then passes through a diffraction grating in the same orientation as the slit to diffract the various wavelengths of light in different directions. Finally the light hits a standard optical sensor for the wavelength range in question.
To obtain a full image from this kind of camera, one typically moves the object that is to be imaged, either by placing it on a moving stage or by placing it on a conveyor belt.
Is is important to note that a hyperspectral image has more channels than a traditional RGB image, hence it is often described as a “hyperspectral data cube”.
Imaging Setups
In order to measure an actual image, pushbroom cameras require the user to create motion in the direction perpendicular to the slit and to acquire multiple frames during that motion. The frames then have to be combined to an image (or rather a HSI data cube). It should be noted that the aspect ratio of the image that is generated by the motion of the imaged object relative to the camera is only correct if the speed of the motion is synchronized with the frame rate of the camera. Otherwise the image may appear streched or squished.
There are two types of setups: desktop systems, typically used in a lab environment, where the measurement process is a manual process, and industrial setups, where the measurement is performed automatically.
A desktop measurement setup will typically consist of an assembly that contains a camera looking down on a linear stage that is controlled by the software used to record the image. For example, the LuxFlux PolyScanner Inspection System consists of a linear stage, a light, a hyperspectral camera, an industrial PC, and the fluxTrainer hyperspectral data acquisition and analysis software suite.
In an industrial environment the typical setup is above a conveyor belt that transports the objects that are to be measured. In that case the measurement is a continuous process in which the camera constantly images the line that is currently beneath it in order for the software to reconstruct an endless image. Data processing has to occur in real-time every time a new line is being recorded. LuxFlux provides embedded real-time data processing software.
Lighting
The most underestimated part of performing hyperspectral measurements is how to provide a good lighting source. Note that as part of the PolyScanner Inspection System, LuxFlux provides a light source that fulfills all of the required criteria.
Broad Spectrum
Many light sources emit only a set of various peaks at specific wavelengths. LED lights are good enough for the human eye or a standard RGB camera to pick up a somewhat accurate picture, but they are not good enough for a hyperspectral camera. A camera only measures the light that is reflected by an object (under the assumption that the object does not emit light by itself), and if the light used to illuminate the object does not contain a specific wavelength, the object will not reflect it, and the camera will not pick up that specific wavelength.
For this reason it is imperative to use a light source that emits the entire spectrum that is to be imaged, in the ideal case with equal intensity.
Halogen lights provide a broad spectrum of comparable intensity in the short-wave infrared regime (up to 2500nm). That spectrum also extends down into the visible spectral range. For the visible range there are specialized LEDs that emit broad-spectrum light.
For this reason it is recommended to use Halogen lights to illuminate samples when using a SWIR camera (900nm - 1700nm), and a combination of Halogen lights and broad-spectrum LEDs when using a VNIR camera (400nm - 1000nm).
Light Intensity
A hyperspectral camera requires a higher light intensity when compared to a regular camera for the same scene. The reason is quite simple: if light hits a single pixel on a regular camera, the same light is split up into its wavelengths components on a hyperspectral camera. A pixel of a hyperspectral camera therefore only sees a fraction of total light of the point in space that is projected onto it.
For example, a pushbroom camera that splits up into 300 bands will see 1/300th of the light per pixel as compared to a regular camera that images the same point in space with just a single pixel.
Homogeneity
When imaging a scene the light applied to the scene should be roughly homogeneous across the entire scene. In the case of a push broom camera, for example, the entire line that is being imaged should be illuminated with approximately the same intensity. While this is true for computer vision in general, not just hyperspectral measurements, this is more difficult to achieve when performing hyperspectral measurements, due to the other criteria the light has to fulfill at the same time (broad spectrum, larger overall intensity).
Shadow Free
Finally, directional light produces shadows and highlights. These features affect the spectroscopy analysis of data negatively. Hence, it advisable to use a diffuse light source that reduces the shadowing.
Reference Measurements
While white balancing can be important in regular computer vision applications, it is especially important in hyperspectral applications. Hyperspectral imaging unleashes its full potential when spectroscopic methods can be applied to indivdiual pixels. For example, in the SWIR wavelength range, hyperspectral imaging may be used to distinguish various types of plastic from another. But spectroscopic methods rely on an accurately calculated absorbance, that is the fraction of light that is absorbed by the material. The amount of light that enters the camera is determined by both the lighting conditions and amount of light a material absorbs. A white reference measurement is the simplest method of removing the influence of the lighting conditions themselves, so that (in an ideal case) only the material properties themselves influence the amount of light the camera sees.
A white reference measurement should be performed in the same lighting conditions as the real measurements, with the same camera parameters (exposure, gain). The user should place a white reference of suitable material underneath the camera at the same distance as the objects that are being imaged and perform a number of measurements that should be averaged to reduce the noise (typically one would average 10 or more measurements). The intensities of the actual measurements may then be divided by the intensities of the reference material to remove the influence of the lighting conditions, which also takes care of remaining inhomogeneities in the illumination.
As lighting conditions may change slightly over time, it is recommended to perform a white reference measurement as often as is feasible. For a lab environment a white reference measurement should be taken at least at the beginning of every measurement series. For the most accurate measurements the white reference measurement should be repeated in regular intervals, for example every 15 minutes.
Some light sources may require some time to reach an equilibrium state once they are switched on. During that time the spectrum of the light they emit may change slightly. For this reason it is considered best practice to wait some time after switching on the light source before performing any measurement (and especially before performing the white reference measurement). The warmup time will depend on the specific light source, but waiting for at least 5 minutes is good practice.
Note that the subtler the difference in spectra, the more important a good white reference measurement becomes. For example, when distinguishing PE from PP with a SWIR camera, the white reference does not need to be perfect, as the spectral differences are quite prominent, and one could reuse a white reference measurement made on a different day, for example. On the other hand, to accurately measure substance concentrations, or perform precise color measurements, an accurate white reference measurement is required that faithfully captures the lighting conditions.
Calibration Files
Most HSI cameras come with calibration files. There are two types of calibration information that applies to HSI cameras:
Pixel to wavelength mapping: since a point in real space is mapped to an entire line of a push broom camera, with different wavelengths being mapped to different pixels, it is imperative that the software processing the data knows how which pixel corresponds to which wavelength. While the overall wavelength range will depend only on the camera model itself, manufacturing tolerances will dictate the precise values of each pixels and must be determined, which is typically done by the manufacturer.
Correction information: for all cameras there will be aberrations due to the optics involved. If these corrections are to be performed in software, the required information must be provided in some manner.
Note that this correction information is not always provided to the software processing the data: some cameras will perform the required corrections on the camera itself (typicall in an FPGA), while other cameras are constructed in such a manner that the aberrations are small enough not to matter in most use cases. The only information that a software processing HSI data from a camera must always have is the mapping between pixels and wavelengths.
There is currently no standardized format for calibration files for HSI cameras, each manufacturer has their own format they support.