Digital Color Imaging in Biomedicine

1. Introduction

One of the main applications of tele-medicine is medical examination or health care advisement via network systems. For this purpose, the quality of the medical examination through a tele-medicine system must be maintained, for example, the quality of patient images with color information needs to be ensured, in addition to the image acquired by various kinds of methods. Current color imaging systems, however, do not reproduce images with the original color of an object, mainly because of the following reasons;
(1) The characteristics of color imaging systems are device-specific.
(2) The spectral sensitivities of color cameras are different from those of human vision.
(3) When the illumination environments of the image capture and observation sites are different, the difference is not properly corrected.
To achieve high-fidelity reproduction of the original color of an object in tele-medicine systems, we have developed a system in which accurate color information is acquired using a multispectral camera along with measurement of the illumination light spectrum. In addition, a system suitable for clinical use was developed, and the data format for storage and transmission is also investigated in the project directed by Medical Information System Development Center (MEDIS-DC) for 1997-1998.
This report outlines the development of and future problems in color reproduction systems for medical applications.

2. Color image capture and display

In the input of color images, it is needed to acquire tri-stimulus values in the chromaticity coordinates, which are given by the characteristics of human color vision, for correct color reproduction. However, it is not possible with conventional color cameras, because of the reasons presented in the previous section.
In addition, we have to consider the difference of the illumination environments at the image capture and observation sites. Human vision changes its spectral sensitivity depending on the illumination environment, which is called color adaptation. Therefore, if an image captured under a fluorescent lamp is displayed under an incandescent lamp, the whole image may be seen as bluish, because the observer's vision is adapted to his/her environmental illumination. On the other hand, the spectrum of reflected light changes according to the spectral intensity of light illuminating the object. The observer recognizes a white object to be white, on account of the balance of the illumination change and the color adaptation of human vision.
Therefore, two kinds of methods are needed in color reproduction, as follows;
(1) Reproducing a color as if the object is being observed directly. This is equivalent to the observer moving to the image capture site.
(2) Reproducing a color as if the object is under the illumination environment of the observer.
When the illumination environment of the image display is different from the light illuminating the object, the color reproduced by method (1) differs from that when the object is actually observed, due to color adaptation. Based on method (2) it becomes possible to reproduce the color as if the patient exists in front of the physician.
In order to reproduce the color as if the object is in the observation environment like (2), the component of the illumination spectrum of the image capturing environment is removed from the acquired image data, and the spectral reflectance of the object obtained is multiplied by the spectrum of observation illumination, so that the chromaticity under the observation environment is calculated. At this point, it is necessary to acquire the spectral reflectance of the object with good accuracy, to find the color of the object under arbitrary illumination light, using the following methods;
* Capturing a multispectral image with a multiband camera.
* Estimating the spectral reflectance from the captured image using the statistical characteristics of the object class.
We can utilize the statistical characteristics of the spectral reflectance of objects, for instance, the spectral reflectance of skin in dermatology, and mucous membrane of gastric or colon surface in endoscopy. In pathology, the spectral transmittance of stain is also available. It is known that the spectral reflectance of the skin surface and the mucous membrane can be described by the combination of small number of basis functions [1,2,4], so that the number of bands of the camera can be reduced without considerable loss of accuracy.
In the image display system, the calibration of the display device is necessary for natural color reproduction. It is expected that color management technology, which has remarkably progressed recently, will be applied to make it easy to do the calibration procedure. Furthermore, in the image transmission and storage system, it should be noted that non-reversible image compression does not preserve color accuracy, because color accuracy is not thoroughly considered in conventional image compression techniques. The technology for image compression suitable for high-fidelity color reproduction is expected to be developed.

3. Color reproduction system

In this section, let us introduce a system based on a multispectral camera with rotating filters (Olympus Optical Co.). This camera can acquire 10-band multispectral images at three-frames per second, by rotating a tablet where ten narrow-band interference filters are attached.
At first, a multiband image is captured with the multispectral camera, along with measurements of the spectrum of illumination light for the object. To transmit the color image, the following two kinds of techniques are available according to the situation.
(1) When the illumination environment of the observation site can be specified by the sender.
If the image capture site receives the illumination spectrum of the observation environment beforehand, it then calculates the color image expressed by chromaticity under the observation illumination environment. In this case, an image expressed by three bands is transmitted, so that the amount of transmitted data does not increase considerably as compared with conventional RGB color image transmission.
(2) When the illumination environment of the observation site cannot be specified.
If the image is sent to several sites, or is stored for later reproduction, it is difficult to know the observation illumination spectrum in advance. To get the best color reproduction under an arbitrary illumination environment from the acquired image data, the multispectral image data should be sent along with the information such as an illumination spectrum and the characteristics of the camera. At the image display site, the color image is reproduced while correcting the characteristics of the display device, after the image is converted to the one under the observation illumination environment in case of method (2).
In the experiment, the spectral reflectance of the skin (484 samples) was measured with a spectrophotometer to find the statistical characteristics of the object. Then the contribution rate of the principal components is calculated by principal component analysis. As a result, it was found that the spectral reflectance of the skin can be well described by 6-8 basis functions, and that high-accuracy color reproduction is possible with a 6- or 8-band multispectral camera. In addition, it was confirmed by the experiment using a 10-band multispectral camera that the accuracy of color reproduction is almost comparable with the color discrimination ability of human vision.
The illumination spectrum of the image capturing environment, which can be measured with a spectrophotometer, can also be obtained by taking the image of a color chart. Namely, the transfer function of the image capturing system including the illumination spectrum is estimated from the image of a color chart that has several color patches of known spectral reflectance. In the case that arbitrary objects are being considered, the color chart should have many different color patches, but the number of color patches can be reduced if the variation of the objects can be limited within a class of known statistical characteristics [2]. For example, for the color reproduction of the human skin, a color chart that is specially designed for skin imaging is also captured with the object, so that the illumination conversion becomes possible. Furthermore, camera calibration can be also done using the image of the color chart, even if the spectral sensitivity of the camera is unknown.
In actual clinical practice, it is sometimes necessary to use a commercial digital still cameras from the aspects of easy operation, lower cost, and compact size. However, correct color reproduction is not guaranteed by commercial digital still cameras because the spectral sensitivity of the camera is different from that of human vision, and as well, image processing is applied for preferable (not natural) color reproduction. To make use a digital still camera for natural color reproduction, an accurate color reproduction system must have been realized by modifying the internal processing of the camera and applying a calibration method that uses the color chart. Nevertheless, note that still there is a limit in the color reproduction accuracy possible with a 3-band camera.

4. Applications of color information

In the color reproduction system mentioned above, color reproduction under various illumination environments is achieved by measuring colors. Measured color information can be used not only for the transmission, but also for the examination of temporal color changes by saving the color data, or for reference as a color image database. For example, it will be possible to detect minute changes of blood flow or dermatological disease by comparing the colors of time-sequence images. Moreover, the measured color information can be applied to the quantification of lesions or even to diagnosis supporting systems, by elucidating the relationship between the color and the condition of the disease. In endoscopy, measurement and analysis of the spectral reflectance of the mucous membrane of the stomach and colon are being done with using an endoscopic spectroscopy system to investigate the relationship between the lesion status and the color [5].
For color reproduction in tele-medicine or a database for the utilization of color information, the image data must be transmitted or stored in a form that can be utilized as quantitative information. In the DICOM standard of medical image transmission, the format for natural color reproduction is not implemented in the standard for visible light images. For this reason, a data format for color reproduction under various illumination environments is being developed by the "color reproduction in medicine" working group directed by MEDIS-DC, as shown in Table 1.