Medical images visualization with python

Utilities for exploratory data analysis and visualization on nifti files (Colorectal cancer) with python

Data visualization on medical images

Medical images such as MRI or CT scan are 3D objects often difficult to visualize in publications. One solution is to create a 3D representation with all 2D slices. Another solution, addressed here, is to plot selected 2D slices among all image stack and their corresponding tumor masks.

First, we need verify if there is no problem in the dataset :

• Do all images/masks are in the same size?
• Do all images have a corresponding mask?
• Are masks binary ? (0 : pixel do not contained tumor, 1 : pixel containing tumor)

(Here we do not addressed issues about image preprocessing including intensity normalization, noise and contrast…).

We are using 126 3D nifti files corresponding to computerized tomography (CT) scan of patient with colorectal cancer. This dataset was initially used at a Medical Segmentation Decathlon for segmentation tasks.

More information and link to download this dataset can be found Here

The job is quite straightforward here as all images already have consistent 512/512 pixels size with corresponding binary mask. But do we have the same number of images per patient? And do we have equal number of image/mask with and without tumor?

Let’s see that. We just need to quantify the number of completely black masks (= no tumors). For later use, we also created a function get_index_tumor_slice returning the index of images/mask with tumors.

Then we can collect number of slices with and without tumors for each patient into a dataframe :

and plot the data :

As we can see at these graphs, most patients have around 100 slices, but only 10% of total images actually contain a tumor. I guess radiologists have to scan the entire abdominal region to find the tumor, resulting in this low ratio.

That would be a problem to train a deep learning algorithm for segmentation/classification, as the number of positive/negative classes in this case should be similar.

For our visualization task, this is just a fact for us to notice, not affecting the results.

To have a global idea of the aspect of 3D object, We first need to visualize individual 2D images from the first slice to the last one with equidistant
intermediary slices (we set the max number of images displayed by 18). We used the custom function plot_image on (example on patient 0050_Colon).

PlotImage(images = image_3d, masks = mask_3d, dislay_mode='mask')

We obtain the images below:

We can see the scan begins below the thigh and up to the lungs covering all abdominal region. The colorectal tumor is somewhere between these slices. We actually can spot it by plotting the corresponding tumor mask by setting argument display_mode =’mask’ of our custom plot_image function to see tumor.

The tumor seem to appear from around slice 57, and finish around slice 89. At this point, we know where the tumor is, but we might want to have a better
look on it. We can set the argument of plot_image_show_tumor_only = True to only display images containing tumors from the first slice to the last one with
equidistant intermediary slices. This way we can see the global shape of the tumor along all the slices :

We might want to spot the tumor in another way. For instance, we can see the same slices and visualize the contour of the tumor mask, overlapping it on the images:

We can also retrieve tumor’s mask bounding box coordinates and plot it on corresponding images :

To finish, we might want to focus on one tumor slice, most likely the one where the tumor is the biggest. To spot it, we just need to find the the mask with the most positive events. We can then plot the images with the bounding box, with the corresponding cropped images around the tumor and finally just the tumor.

# get index slice with the biggest tumor size
index_biggest_tumor = biggest_tumor(image_3d, mask_3d)
plot_tumor(image_3d, mask_3d, index_biggest_tumor)

Optional

Now we can combine all graphs into a publication-like figure:

Figure 1:Proportion of number of slices by patient (A) and total distribution of slices containing tumors or not from 126 patients with colorectal cancer. Example of slices of tumors with contour on one patient (B). Focus on the slice with the largest tumor slice © on patient represented in B.

Final note

There is much more we can do to visualize medical images that is not covered here, like 3D reconstitution and image preprocessing, as well as the development of advanced techniques with amazing library. Not to mention tumors segmentation/detection task this dataset was originally made for…

Still, basic visualization of the dataset is a good start before implementing complex machine learning solutions.

Thanks for reading !

Feedback are welcome and will be appreciated.

Github repository including utility function can be find here