多光谱滤波器阵列的去马赛克研究

The demosaicking is a digital image processing conducted to restore a full color or waveband images from the deficient outputs from an image sensor plated with a color filter array (CFA) or multispectral filter array (MSFA). In this proposal, a new deep learning based MSFA demosaicking is studied to train a deep neural network on a large corpus of images instead of using hand-tuned filters. We create the training set from millions of multispectral images by generating the corresponding mosaicked arrays leaving out two color channels per pixel. We then build a convolutional neural network and train it in an end-to-end fashion. This convolutional network enables discovery of natural correlations in the data. The proposed deep neural network can significantly improve the quality of demosaicking. It can resolve even challenging situations that usually result in zippering or moiré artifacts. The demosaicking algorithms we develop in this proposal can be used in several approaches such as: remote sensing, medical image processing by doctors, fruit Inspection by farmer, defect detection by factory, video surveillance by police. This project provides certain technical support for medical, agriculture, and other industries, thus has important academic significance and practical value. Besides, it has considerable economic benefits to be widely used in future camera industry and even space- and ground-based observations.

去马赛克方法是一种通过具有色彩滤波阵列（CFA）或多光谱滤波阵列（MSFA）的图像传感器所得匮乏输出恢复全彩或全谱段图像的数字图像处理技术。本项目研究一种新型深度学习MSFA去马赛克方法在大数据图像集上训练深度神经网络以代替人造滤波器，首先通过每像素去除双色彩通道生成对应马赛克阵列创建数百万多光谱图像构成的训练集，构建首尾相连训练的卷积神经网络以利用数据潜在相关性，所提出的深度神经网络能够明显提高去马赛克质量，解决zippering或moiré伪影所产生的技术性难题。本项目所研究的去马赛克技术能够应用于遥感成像、医学图像处理、水果分级监测、工业缺陷监测、警用视频监控等方面，为医疗、工业、农业等都提供了一定技术支持，因此具有重要的学术意义和使用价值；并且能够广泛应用到未来相机行业和天地基观测中，具有可观的经济效益。

传统去马赛克方法通过具有色彩滤波阵列（CFA）或多光谱滤波阵列（MSFA）的图像传感器所得的匮乏输出来恢复全彩或全谱段图像。本项目主要结合深度学习方法和MSFA技术提出一种新型可复原多光谱图像的神经网络，以代替人造滤波器去除图像马赛克。实验证明，该方法能有效解决zippering或moiré伪影所产生的技术性难题。并且基于本项目，提出了在高光谱图像的无损压缩和传输方面的基于聚类差分脉冲编码调制的算法，有效缓解了高光谱图像的存储、传输和处理压力。在极光光谱数据的无损压缩方面提出了一种基于预测的在线差分脉码调制方法，得到极光光谱图像的平均压缩时间为0.06s。基于深度学习的去马赛克方法克服了传统方法在处理大数据时的效率低、类型单一的问题，该技术在遥感图像、极光图像、医学图像等方面都有着广泛的使用价值。