基于光谱和图像特征融合的苹果内外部多品质指标同步在线检测方法研究

Today, the external and internal qualities of apples are detected independently in an online sorting system. The detection accuracy could not be improved without the effective fusion of spectral and image features. In this project, a method for online simultaneous determination of multiple internal and external quality parameters of apples based on spectral and image features fusion was proposed for the detection of common surface defects, soluble solid contents (SSC) and firmness in ‘Fuji’ apples. Firstly, the hyperspectral images were acquired after optimizing acquisition parameters and spatial layout between light source and the detector of a 400~1000 nm line-scan hyperspectral imaging system. The absorption and scattering effects between light and different apple tissues were studied and the optical properties parameters including absorption and reduced scattering coefficients were determined. The chlorophyll, carotenoid and anthocyanin content are measured and the spectral indexes for these pigments are determined. Then, the size, color and texture features were extracted from a region of interest (ROI) in hyperspectral images. These features were integrated with pigments spectral indexes, optical properties parameters, diffuse reflectance spectra to select the effective wavelengths and image features set for common surface defects (including bruising, rottenness, russeting, insect bites, cracky fruit, sun burn and rubbing), SSC and firmness detection based on chemometrics, pattern recognition and image processing methods. The algorithms for different defects classification and models for SSC and firmness prediction were established. Finally, a prototype online detection system was developed. The multispectral images were acquired directly from the line-scan hyperspectral imaging system according to the selected effective wavelengths. The acquisition time and computation load was decreased dramatically with less number of effective wavelengths. So the acquired multispectral images could be processed in real-time for simultaneous determination of defects, SSC and firmness.

当前苹果内外部品质的检测是独立进行的，光谱与图像缺乏有效的融合以提高检测精度。本项目以富士苹果的常见表面缺陷和糖度、硬度为检测目标，研究光谱与图像特征融合的苹果内外部多品质指标同步在线检测方法。通过优化 400~1000nm 的线扫描高光谱成像系统的采集参数，获取苹果的高光谱图像数据。研究光与苹果组织作用时的吸收和散射效应，获取光学特性参数。对苹果表皮的主要色素含量进行检测，确定有效检测色素含量的光谱指数。从高光谱图像中的感兴趣区域提取尺寸、颜色、纹理等特征，融合色素光谱指数、光学特性参数、漫反射光谱，采用化学计量学、模式识别和图像处理方法优选出用于苹果损伤、腐烂等缺陷识别及糖度、硬度检测的特征波长和图像特征集，建立缺陷分类模型和糖度、硬度预测模型。开发在线检测原型系统，从高光谱成像系统中直接提取与特征波长相对应的多光谱图像并进行实时处理，大幅度降低检测所需的时间，实现多指标同步检测。

本项目以富士苹果的常见表面缺陷和可溶性固形物含量的在线检测为目标，研究光谱与成像方式下苹果内外部多品质指标在线检测方法。在苹果内部品质检测方面，搭建了苹果光学特性检测系统，揭示了水果自身特性差异及环境因素对检测模型影响的规律，提出了果皮果肉双层检测理论模型，研究了样品特性及环境因素如尺寸、姿态、颜色、速度及采样方式等对检测模型影响的补偿方法，进一步研究了不同年份模型的优化升级及模型传递方法，实现了苹果全透射光谱的有效获取及实时处理，进一步实现苹果品质分布的可视化检测。在苹果外部缺陷检测方面，研究了不同缺陷的光学特性，揭示了缺陷与正常果皮之间的差异，提取了不同缺陷同步检测的关键特征波长，提出了基于多光谱成像与深度学习融合的苹果缺陷检测方法，解决了苹果表面多缺陷同步检测的难题，同时实现了缺陷与果梗/花萼的区分。在上述基础研究的支持下，开发了全透射多光点光谱苹果内部品质在线检测系统、基于多光谱成像与深度学习融合的苹果缺陷在线检测系统、基于高/多光谱成像的苹果内外部品质在线检测原型系统，为苹果产业采后商品化处理水平的提升提供了技术支撑。