Rapid phenotyping

Rapid phenotyping of prawn biochemical attributes using hyperspectral imagingStuart Hinchliff supervised by Professor Ronald White and Professor Dean Jerry

IndexAimsBackground informationTechniques applied and resultsPreliminary statisticsUI demonstrationOutcomes and future work

MotivationThe investigation of hyperspectral imaging as a fast and non-invasive technique that could lead to improved selective prawn breeding programs.

Aims:To obtain hyperspectral image data on a variety of prawns.To investigate appropriate image processing methods for distinguishing prawns. To prepare a statistical environment for correlating prawn spectra to biochemical attributes, and training models.To structure the explored techniques in an intuitive user interface.

Hyperspectral ImagingTypical RGB image (eg. normal image in formats such as .jpeg) is made up of 3 bands:

Multispectral images and hyperspectral images have many more than this (our images have 240 bands ranging from ~ 400 nm to 1000 nm).Average image size: 800 MB

Hyperspectral ImagingData is made up of many images overlayed. Each image is called a band:

NIRS (Near-Infrared Spectroscopy)Higher energy than mid-IR and therefore is useful in probing bulk material with little to no preparation (water is reasonably transparent in NIR)Region of electromagnetic spectrum: 700nm to 2500nmComplex spectra due to molecular overtone (harmonics) and combination vibrations to extract chemical information, multivariate calibration techniques are used such as:Principal component analysis,Partial least squares, andNeural networks.

Obtaining the spectraRemove undesirable background elements (tray, label and rubber band)Two approaches: using traditional RGB methods, using hyperspectral techniques

ConversionBandLabel

RGB TechniquesConsidered due to low volume of data, therefore high speed and efficiencyThresholding and morphological operations:

Advantages: Very efficient with reasonable accuracyDisadvantages: Different lighting conditions would require calibration of threshholding, missing information.

Example 1Example 2

RGB Techniques continuedClustering:

Advantages: Low use of resources, more robust than morphological operationsDisadvantages: Lower accuracy (doesnt distinguish labels), more clusters significantly increase computational time, clustering is randomised

Conversion to L*a*b space before using kmeansConversion to chromaticity space before using kmeans

ScyllarusScyllarus is hyperspectral software developed by NICTA A C++ API and a MATLAB toolbox are available.

Advantages: Uses advanced algorithms to pre-process images and identify materials. These materials could be useful for identifying trends.Disadvantages: Poor efficiency (perhaps C++ API could be used).

Neural NetworksSupervised machine learning technique that adjusts weights to ensure inputs match outputUses nested cross-validation (training, test and validation data) to optimise algorithm for the data and avoid overfitting

Neural NetworksSupervised training on a manually classified image.

Advantages: Very accurate, is robust and can be improved with further trainingDisadvantages: Not as efficient as other methods, noisy pixels will be misclassified

Statistics and AnalysisEnvironment developed to obtain prawn spectra signatures from images, visualise trends (principal component analysis), preprocessing (scaling) and train on a factor (neural networks).

Statistics and AnalysisThe investigation has demonstrated that no trends exist for training on area, however we are optimistic for other parameters.

Software Demonstration

OutcomesSoftware has been developed as an all-in-one package for reading data, identifying prawns and analysing trends.Scyllarus uses advanced preprocessing to identify different materials in the images could be extremely useful in analysing segments.Solid foundation for future work. Could be a suitable honours project.

Future WorkTraining on the biochemical data.Additional settings and features for user to tweak to aid in training/visualising trends.Training on a particular wavelength could remove the need of slow hyperspectral imaging process.Perhaps switching languages and using the C++ API due to its improved efficiency.