Multispectral Imaging: More Than Just A Picture

Ray LeerentveldDirector of Product Management

Lumenera Corporation

Presentation Outline

• What is multispectral imaging (MSI)?• How does it differ from hyperspectral imaging?

• MSI acquisition

• Recent advancements in multispectral technology

• Multispectral Imaging Techniques

• Multispectral applications:• Case Study on Normalized Difference Vegetation Index

• Food Inspection

• Textile Inspection And Analysis

• Printed Circuit Boards

What Is Multispectral Imaging?

• Multispectral imaging involves breaking the light spectrum into multiple bands beyond the basic RED, Green and Blue

• Multispectral can run beyond the visible spectrum into Ultra Violet or out to Infra Red

Content Courtesy of Midwest Optical Systems

What Is Multispectral Imaging?

• Multispectral imaging is a subset of Hyperspectral imaging typically seen as 2 to 32 segmented bands

• Hyperspectral can have a hundred or more bands

Multispectral Hyperspectral

Content Courtesy of Midwest Optical Systems

Multispectral Vs. Hyperspectral Imaging

• Multispectral can often be represented with a single false-colored image to provide quick visual representation of the data

• Hyperspectral utilizes a three dimensional dataset known as a hypercube that requires 3D analysis to determine the spectral data of interest

Multispectral Image Acquisition

• Multispectral image acquisition can be accomplished using:• Wafer-level coated sensors

• Lighting with varying wavelengths

• Multispectral Camera

• Specialized Filters (Multi-Camera) or Filter Wheels

Content Courtesy of Delta Optical Thin Film and Glana

Multispectral Imaging Acquisition – Sensors

• A variety of coatings can be applied directly on the wafer in arbitrary patterns to suit a specific application’s needs

• Pixel level ----RGB+NIR

• Area ------------Mosaic of squares at different wavelengths

• Pros: Can be used on small sensors, highly customizable, built into the sensor, easy to integrate into cameras

• Cons: Expensive in small volume, strong side bands, and limited spectral complexity

Multispectral Imaging Acquisition – Lighting

• The subject can be illuminated with varying wavelengths of light and images captured with a monochrome camera

• Pros: Lower cost and higher resolution

• Cons: Static subject, limited to diversity of light wavelengths, requires controlled environment

Multispectral Imaging Acquisition – Camera

• Cameras with special gratings & sensors can be used for MSI

• Pros: accurate and purpose-built

• Cons: large, low sensitivity: ~25µm entrance slit, and expensive

Content Courtesy of Delta Optical Thin Film and Glana

Multispectral Imaging Acquisition – Filters

• A number of filter setups exist to achieve multispectral imaging:• Rotating filter wheel

• Pros: Large number of filters available, high resolution, and lower cost

• Cons: Static Subject, large filter wheel assembly

• Single, dual, and multi bandpass filters using multiple cameras• Pros: Application-specific filters, high resolution, and lower cost

• Cons: Quantity of Cameras required proportional to wavelengths

Recent Advancements In Multispectral Tech

• Delta Optical Thin Film’s continuously-variable sensor-mount hyperspectral filter

• Bandwidths and frequency range are customizable

• Turns a high-resolution 35mm format camera into a hyperspectral system

• Spectral resolution directly proportional to spatial resolution

Content Courtesy of Delta Optical Thin Film and Glana

Multispectral Imaging Techniques

• Step and Stare • Subject must be static

• Snapshot then cycle filter/lighting/etc

• Push Broom/Panning• Subject/camera must be moving linearly

• Sequence of images are stitched together to form hypercube

Spectral Results

Panning IllustrationContent Courtesy of Delta Optical Thin Film and Glana

Camera Solution Filter Versus Grating

• Shorter optical path smaller solution

• More light reaches the sensor

• 3D hyperspectral imaging possible due

to linear camera panning

• Image stacking in the cube can be easily

accomplished due to high spatial

resolution

Content Courtesy of Delta Optical Thin Film and Glana

Why Is Multispectral Imaging Important?

• See more than the naked eye: See beyond color

• The evolution of imaging MonoColor3DMultispectral

• A wide range of industries benefit from multispectral imaging

False-color representation of a hypercubeContent Courtesy of Delta Optical Thin Film and Glana

Case Study

An Example of Multispectral Customization

Example Of Multispectral Customization

• Normalized Difference Vegetation Index (NDVI)

• Started with multispectral satellite imaging

• Applied the technology to a multi-camera earth-based solution

• Evolved to a single camera solution

• Takes advantage of an existing, well defined technology and applying it to a custom solution

Cost

Normalized Difference Vegetation Index (NDVI)

• NDVI is a popular index to measure healthy vegetation using satellite imagery

• Normalized between -1 and +1• Healthy vegetation ranges from 0.2 to 0.8

• Plants absorb red light through photosynthesis and reflect NIR light

• NDVI Equation: 𝑁𝐷𝑉𝐼 =𝑁𝐼𝑅 − 𝑅𝑒𝑑

𝑁𝐼𝑅 + 𝑅𝑒𝑑

Vegetation Spectral Response

• Review Hyperspectralresponse to select targeted Multispectral bands

• Healthy plants absorb blue and red, reflect green and highly reflect NIR

• Single color camera with dual bandpass filter can be used to approximate NDVI

Content Courtesy of: “Low-cost multispectral vegetation

imaging system for detecting leaking CO2 gas”

Applied Optics. 51, A59-A66 (2012)

Single Camera NDVI

• A pseudo-NDVI camera is created by placing a dual bandpass filter in front of a color camera

Content Courtesy of Midwest Optical Systems

𝑁𝐷𝑉𝐼 =2 × 𝑅𝑒𝑑 − 𝐵𝑙𝑢𝑒

𝐵𝑙𝑢𝑒

*Green channel not used

Single Camera NDVI

• Can be applied to food inspection to search for inorganic contamination or detecting unhealthy produce

NDVI Approximation Using Lumenera’s Lt965C and MidOpt’s DB475/850

Food Inspection

• Human inspection is predominant

• Since the early 2000s there has been a significant uptick in research surrounding multispectral food inspection

• Main areas of research include:• Fruit – Damage & disease detection

• Vegetables – Damage & shelf life determination

• Meat – Bacteria detection & marbling quality

• Seafood – Determine expiry & gelatin adulteration

Food Inspection – Light Penetration In Fruit

• Differing wavelengths will penetrate fruit at various depths for example in an apple

• 4mm between 700 and 900 nm

• 2 to 3 mm between 900 and 1,900 nm

• Multispectral imaging can analyze structures beneath the surface of fruit, helping to detect damage and disease

• Reference libraries can then be created to catalog and reference good vs bad produce or a specific property

• Red color of Cranberries related to the anthocyanin concentration, an antioxidant

directly related to ripeness and value

Textile Inspection – Matching Of Materials

• Multispectral cameras can image textiles with more color accuracy

• Samples can be accurately reproduced• Various parts of a garment can be accurately color matched

• Leather, vinyl, plastic, thread, metal, polyester spectral match to fabric

• Specific tint reproduction can be conserved and exchanged

Printed Circuit Board Inspection

• Typically inspected with RGB & IR for thermal functionality

• Large variety of components makes inspection challenging & time consuming

• Component and precious metal detection is essential for recycling

• Multispectral imaging can image buried copper

PCB Inspection – VIS Vs. NIR

VISNIR

PCB Inspection – VIS Vs. NIR

VIS NIR

Summary

• New filter technologies offer many approaches to MSI• Lower Cost

• Highly Customizable

• High Spatial and Spectral Resolution

• There are a number of ways to customize MSI using:• Filters

• Cameras

• Sensors

• Lighting

• Many applications can benefit from MSI:• Food Safety/Inspection

• PCB Inspection

• Textile Inspection

Acknowledgements

• Supplemental Content Providers

Contact Information

Ray LeerentveldDirector of Product Management

Lumenera Corporation7 Capella Court

Ottawa, Ontario

Canada

Telephone: +1 613 736-4077

Email: ray.leerentveld@lumenera.com

www.lumenera.com

Booth: 2662