Methods for the application-optimized design of multispectral image sensors, using the example of an innovative multimodal camera approach for optical dermatoscopy

Abstract

The machine-assisted diagnosis of skin cancer is now considered state of the art. As a rule, proven high-resolution color (RGB) camera technology is used for optical imaging of the skin, and the data obtained is analyzed using traditional methods or modern image processing techniques based on neural networks. This article presents methods to optimize the multispectral design of multimodal camera systems for that purpose. Since multispectral optical features are expected to be of crucial importance here, the focus is on multispectral imaging of skin areas. To this end, a new multispectral snapshot camera with an innovative multi-aperture approach (MAC) is used, combined with other imaging techniques to capture color texture and 3D information. To identify an application-optimized design, universally applicable algorithmic methods are presented that enable automatic configuration based on available spectral information regarding the application class and the system design, as well as known technological limitations or constraints. Our paper illustrates a holistic approach to image processing, in which the focus is on ensuring the availability of the best possible raw data by optimizing multispectral imaging for a specific class of problems. The proposed methods are evaluated using simulations in realistic scenarios to validate the subsequent implementation steps. The considerations and findings presented are part of the first phase of the funded multiSKIN project, whose goal is to develop innovative multispectral camera technologies as part of a multimodal camera array for optical dermatoscopy.