Important Aspects in Smart Segmentation of Medical Objects of Interest


Segmentation of object is an age old problem in computer vision. Also, it is the first step in many object recognition and tracking tasks. Image segmentation is the process of partitioning a digital image into multiple segments such that each segment make sense of the object of interest. Another view of object segmentation is to detect a closed boundary of object in such way that same label is assigned to each pixel on and inside closed boundary. So, depending on the appearance and scenario of the object, a choice can be made either to detect the set of pixels which share the same characteristics or detect set of pixels forming the object boundary. As an example, Fig. 1 shows nuclear size, shape, density and surface homogeneity variations across four different image views obtained from different breast tissue biopsies [1]. In this scenario, a non-generalised nuclear segmentation designed only for the image in Fig. 1 (a) could fail on the images shown in Fig. 1 (b,c,d).



In medical world, object segmentation is comparatively more critical since a patient is involved whose diagnosis, surgery or treatment can be dependent on the output of the segmentation technique. As example, in breast cancer, the mitosis count in slide of a subject biopsy is important factor that decide whether the subject has breast cancer and the stage of the cancer [2]. Thus, any inaccurate segmentation will affect the patient’s health and welfare. In such a scenario, robust segmentation techniques should be developed where various factors should be considered while designing the algorithm for segmentation. “Generalised” object segmentation is an idea which is on the rise and should be propagated to achieve robust object segmentation [1].

A generalised object segmentation is an object segmentation technique which achieves robust performance in each and every scenario of the object and compensate for the variations such as acquisition, illumination and other domain specific variations. A simple way to achieve robust segmentation is not to choose a hard threshold which is not generalised. Most of the hard thresholds are incapable to account for source, illumination and scenario variations. In complex scenes of medical object segmentation, image processing or rule based methods fail since these methods involve various parameters to be manually defined. An interesting way out would be to make the parameter to be dependent on the image properties or generically learned from the available image data. Another priori information in medical domain is that medical objects are generally structured due to its biological cause. As an example, mitosis nuclei are nuclei undergoing biological division and their DNA gets doubled which results in their larger appearance [2] as shown in Fig 1 (a). It make sense to use this knowledge as a priori information for segmentation. In another case, optical coherence tomography (OCT) images of retina are structured and the relative position of the different layers of retina is a priori information which could be employed for retinal layer segmentation.

Moreover, if thousands of samples of same object are available, deep learning techniques can learn meaningful mapping to segment the object of interest or estimate the segmentation parameters from such rich data. Recently, U-net deep learning framework achieved state of the art performance for medical image segmentation with limited samples [3]. In any segmentation technique, analysis of the data distribution for different types of variations is most important. Ideally rich dataset will have equal samples of each type of variations like source, illumination and scenario etc. Any robust segmentation technique which performs accurately on such rich data, is a “smart segmentation” technique.


[1] Vahadane, Abhishek, and Amit Sethi. “Towards generalized nuclear segmentation in histological images.” Bioinformatics and Bioengineering (BIBE), 2013 IEEE 13th International Conference on. IEEE, 2013.

[2] Roux, Ludovic, et al. “Mitosis detection in breast cancer histological images An ICPR 2012 contest.” Journal of pathology informatics 4.1 (2013).

[3] Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for biomedical image segmentation.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer International Publishing, 2015.

Image credit –