Hyperspectral Pathology Image Segmentation of Cancer Cytoplasm: A New Approach Based on Data Augmentation

In pathological diagnosis, Hematoxylin-Eosin (H&E) staining is a commonly used method for observing tissue morphology. Hematoxylin stains cell nuclei blue-purple, while eosin stains cytoplasm and extracellular matrix pink. Pathologists observe these stained sections under a microscope to judge whether cells have undergone cancerous changes. In the process of determining cancer types, accurate identification of the cancer cytoplasm region is a key step.

In recent years, hyperspectral imaging technology has gradually entered the field of pathology research. Compared with conventional CMOS cameras, hyperspectral cameras can acquire richer cellular information. However, this technology faces two realistic problems in practical applications.

The first problem is the difficulty of data acquisition. The acquisition cost of hyperspectral images is high, and establishing a large-scale annotated dataset is not easy. Yet, deep learning models precisely require a large amount of data to work effectively. The second problem is instrument noise. Hyperspectral cameras based on the linear push-broom mode often generate vertical or horizontal stripe noise, pixel shifts, and other issues during the imaging process. These noises vary according to equipment characteristics and operating environments, making them difficult to avoid completely.

To address these problems, a study was published in the journal Informatics in Medicine Unlocked. They proposed a data augmentation method that utilizes CMOS images to generate pseudo-hyperspectral images simulating real instrument noise, which are then used to train a U-Net segmentation model.

1.Why use CMOS images for augmentation?
A direct consideration is the annotation cost. The texture of cancer cytoplasm in hyperspectral images is complex, and manually annotating them in grayscale images is both time-consuming and laborious. In contrast, CMOS images are visually clear, and their annotation efficiency is much higher. The research team's idea is: use easily annotated CMOS images to generate pseudo-hyperspectral images with noise, and then use these images to train the model.

The specific approach is divided into three steps:
The first step is color conversion and brightness adjustment. Convert the CMOS RGB image into a grayscale image, and adjust the brightness through gamma correction and histogram equalization to make it close to the visual effect of hyperspectral images at a specific wavelength band (such as 580 nm).

The second step is adding instrument noise. Randomly add vertical and horizontal noise lines to the image to simulate the stripe noise generated in actual hyperspectral cameras due to differences in sensor sensitivity or signal processing. The number of vertical noise lines is between 19 and 29, and the positions and widths of horizontal noise lines are also randomized. In addition, information loss and pixel shifts are simulated.

The third step is geometric transformation. Perform operations such as cropping, flipping, and translating on the generated pseudo-images to further expand the dataset.

Through this workflow, 44 original CMOS images generated 792 pseudo-hyperspectral images, which, after being merged with 44 original hyperspectral images, brought the training set to 836 images.

2.How was the experimental effect?
The research team evaluated model performance using 5-fold cross-validation, employing IoU (Intersection over Union) and Dice coefficient as indicators.

When trained with only 44 original hyperspectral images, the IoU was 0.5786 and the Dice coefficient was 0.7304. After adding the pseudo-hyperspectral images, the IoU increased to 0.6458 and the Dice coefficient increased to 0.7820—IoU improved by about 11.6% and the Dice coefficient improved by about 7.1%.

The study also found that image brightness and cell density affect the segmentation results. Darker image types and image types with dense cell nuclei showed relatively lower segmentation accuracy. This indicates that the model has a certain dependency on image quality, and also suggests directions for future improvement.

3. What is the significance of this research?
For the processing of hyperspectral pathology images, data scarcity and instrument noise are two unavoidable obstacles. This study provides an actionable idea: instead of painstakingly removing noise, it is better to let the model learn to deal with noise during the training stage. Generating pseudo-hyperspectral images with instrument noise from CMOS images not only solves the difficulty of annotation but also enhances the model's robustness against noise.

Of course, this study also has limitations. The experimental data came from canine mammary tumor samples, and there were only seven image types. More complex sample types and larger datasets still need to be gradually accumulated.

However, from a methodological perspective, this concept of "using easily annotated images to generate noise-containing training samples" provides a valuable practical path for the application of hyperspectral microscopic imaging in pathological diagnosis. With the gradual popularization of hyperspectral imaging equipment in pathology departments, similar data augmentation methods might help researchers train models more efficiently, reducing dependence on large-scale manual annotation.

Disclaimer
This content is compiled and edited from open academic literature (literature link: https://www.sciencedirect.com/science/article/pii/S2352914826000213). It is only used for industry technical discussion and science popularization learning, and does not make any commercial-related commitments, nor can it be used as an investment reference basis. The various experimental data and conclusions listed in the text will be interfered with by multiple variables such as the test environment, individual differences in samples, and model construction schemes. If applied in actual scenarios, the relevant effects need to be verified through independent testing in combination with one's own scenarios.