Overview / Usage

With the release of CoVid 19 dataset, this model will be extended to work on diagnosis of coronavirus in lungs.

Non-uniform accessibility to healthcare across the country and shortage of qualified healthcare professionals are major barriers to both preventive and curative health services. Furthermore, there is glaring disparity between rural and urban India as regards access to medical diagnosis and other healthcare facilities are concerned. About 55 million people in India suffer from chronic obstructive pulmonary disease , according to a global study, which noted that people of less developed states are more prone to the disease than those living in developed ones. Chronic respiratory diseases were responsible for 10.9% of the total deaths and 6.4% of the disability-adjusted-life-years(DALYs) in India in 2016 as compared with 9.5% and 4.5% respectively in 1990. Of the total global DALYs due to chronic respiratory diseases in 2016, 32% were in India. Fast and Cheap Computer aided Diagnosis will prove extremely beneficial for people in rural India where there is acute paucity of trained radiologists and Doctors per capita. An automated AI system that can reliably identify different disorders from Chest X-rays would be invaluable in addressing the problem of reporting backlogs and lack of radiologists in low-resource settings. Our study will demonstrate that AI based Computer aided diagnosis – deep learning algorithm trained on a large quantity of well-labelled data can accurately detect multiple abnormalities on Chest X-rays. Our end-to-end AI based solution addresses the problem of multi-label thorax disease classification on chest X-ray images. Our model will output the predicted scores corresponding to each of the 14 abnormalities associated with thoracic disease and a class activation map that clearly highlights regions in X-rays characteristic of the different abnormalities. As these systems further increase in accuracy, the feasibility of applying deep learning to widen the reach of automatic chest X-ray interpretation and improving reporting efficiency will add tremendous value in radiology workflows and public health screenings in Indian healthcare sector, particularly rural India, which suffers from acute shortage of trained radiologists and doctors

Methodology / Approach

We have used PyTorch and FastAi library to design our model. We used Chest X-ray 14[1] dataset by NIH for procuring the data for our model.We trained our model on GTX 1080 ti In our approach we have used an ensemble of fine-tuned(Transfer Learning) Xception architecture with DenseNet-121[3]. The fine-tuning is done using state of the art approaches like cyclic learning rate. To address the class imbalance issue we employed recently proposed _ Class balanced loss_ by Yin Cui, Menglin Jia et. al., 2019 Further, the output of our fine tuned transfer learning model is conected to residual attention network for further improving the accuracy. In the previous work by Kapur et. al (CheXNet)[2], all the pathologies are treated equally in classifier learning, i.e., when predicting the labels of each image, all pathologies are given the same weight. Moreover, correlation essentially exists among the labels, for example, the presence of cardiomegaly usually additionally accompanies high risk of pulmonary edema. Therefore, exploring the dependency or correlation among labels could assist to strengthen the intrinsic relationship for some categories. However, considering an individual image, the uncorrelated labels may also introduce unnecessary noise and hinder the classifier from learning powerful features. We wanted to mitigate the interference of uncorrelated classes and preserve correlations among the relevant classes at the same time. To accomplish the above task we will be using combination of residual attention learning and Spatial Regularization Net. Our model uses a category-wise residual attention mechanism to assign different weights to different feature spatial regions. It will automatically predict the attentive weights to enhance the relevant features and restrain the irrelevant features for a specific pathology. Unlike the traditional classification problems, where the prediction can be either correct or wrong, the multi-label classification problem is a more challenging task and requires more special evaluation measures since the performance over all labels should be considered. In a multi-label classification problem, a prediction can be fully correct (all predicted labels are correct), partially correct (some of the predicted labels are correct) or fully wrong (all predicted labels are wrong). We monitored F-beta scores with different values of beta during training and measured AUROC at end of each epoch for validation set. F-beta offers good trade-off between precision and recall. For our problem, F-beta offers more flexibility than plain F1-score as we could adjust the beta value depending upon how much we weigh each of the 14 disease labels. We measured hamming loss and tried different many other loss functions during experimentation.

Importance of our model in Indian scenario

• Non-uniform accessibility to healthcare across the country and shortage of qualified healthcare professionals are major barriers to both preventive and curative health services. Furthermore, there is glaring disparity between rural and urban India as regards access to medical diagnosis and other healthcare facilities are concerned

• About 55 million people in India suffer from chronic obstructive pulmonary disease, according to a global study, which noted that people of less developed states are more prone to the disease than those living in developed ones

• Chronic respiratory diseases were responsible for 10.9% of the total deaths and 6.4% of the disability-adjusted-life-years(DALYs) in India in 2016 as compared with 9.5% and 4.5% respectively in 1990. Of the total global DALYs due to chronic respiratory diseases in 2016, 32% were in India

• Fast and Cheap Computer aided Diagnosis will prove extremely beneficial for people in rural India where there is acute paucity of trained radiologists and Doctors per capita.

• An automated AI system that can reliably identify different disorders from Chest X-rays would be invaluable in addressing the problem of reporting backlogs and lack of radiologists in low-resource settings.

Dataset: The NIH (ChestX-ray14 dataset) comprises 112,120 frontal-view chest X-ray images of 30,805 unique patients with 14 disease labels. In the Chest X-ray14 dataset, each image is annotated with multiple lung-related or heart-related pathologies. If there is no any pathology in an image, it is labeled as “No Finding”. Commonly, Chest X-Ray(CXR) images are labeled with one or more pathologies, which makes the CXR image classification a multi-label classification problem.

• The authors obtained disease labels associated with each image by mining radiological text reports using natural language processing techniques, details can be found in .

• All images were resized to 1024x1024 without preserving aspect ratio.

**Our Aim: **Accurate classification with class activation maps. We intend to create a robust end-to-end AI based solution

that matches or exceeds the competence of expert radiologists and previous Deep Learning models for detection of common thoracic disease labels**(14 abnormalities**) from chest x-rays. Our model will output Class Activation Map that clearly highlights regions in X-rays characteristic of the different abnormalities associated with thoracic disease.

​

A note on reliability of ChestXRay14 dataset

•In the ChestX-ray8[1] dataset which was used for training the algorithm pathology labels were extracted automatically from the radiology reports by text mining. This also means that the ground truth per se is an individual radiologist’s judgment, which in some cases further deteriorated due to the inherent inaccuracies of automated data mining.

• **A frontal chest x-ray is only the beginning: **Already around the time the NIH dataset was released there were concerns about the fact that only frontal views were included. In many cases we use these as standalone tools, but very often we do rely on a very simple yet immensely useful ancillary technique: the lateral view.

• The lateral chest x-ray alone is rarely diagnostic, but in conjunction with the frontal view its effect is supra-additive, by helping clear up not only the localization but the etiology of abnormalities first identified on the frontal radiograph.

• The labels don’t match the images very well. After physical examination of X-ray images, it becomes apparent that many images in this multi-label dataset are inconsistent with the labels.

• The labels are internally consistent, so systems trained on them seem to learn to reproduce them (along with their errors)

• Entity extraction using NLP is not perfect: we try to maximize the recall of finding accurate disease findings by eliminating all possible negations and uncertainties of disease mentions. Terms like ‘It is hard to exclude …’ will be treated as uncertainty cases and then the image will be labeled as ‘No finding’.

• ‘No finding’ is not equal to ‘normal’. ++Images labeled with ‘No finding’ could contain disease patterns other than the listed 14 or uncertain findings within the 14 categories.

Division into Train, validation and test set:

•In our approach, we will randomly shuffle and split data into 60%(training), 20%(validation) and 20%(test). •These correspond to **train.csv, dev.csv, and test.csv **respectively in model_data folder •When working with deep learning, hyperparameter tuning without a validation set and/or cross-validation can easily result in over-fitting.

​

Data Preprocessing:

•Each image is labeled with a one-shot vector L=[l_1, …, l_C], C is 14 in Chest X-ray14. Every element l_C represents the presence of the c_th pathology or not, i.e. , 1 for presence and 0 for absence. •Data pre-processing is meant to adequate our CXRs to the format the pre-trained model requires so we have resized CXRs to the required size 224 x 224 pixels.

•**Image data augmentation **is used to expand the training dataset in order to improve the performance and ability of the model to generalize. •This result in more skillful models, and the augmentation techniques can create variations of the images that can improve the ability of the fit models to generalize what they have learned to new images.

•We augmented the images by using different transformations like horizontal flips, small rotations, warping images around borders, doing affine transformations, and adjusting lighting followed by mean normalization ((x-μ)/σ))

​

•# An example in PyTorch

•tfms = get_transforms(do_flip=True, flip_vert=False, max_rotate=5, max_zoom=1.03, max_lighting=0.03, max_warp=0.03, p_affine=0.05, p_lighting=0.05, )

Proposed solution

• We cast pathology detection as a multi-label classification problem. All images X={x_1,..x_N }, 〖 x〗_i ϵ X, are associated with a ground truth label y_i. Here, y_i is a binary vector y_i ϵ {0,1}^M, where M = 15 (14 abnormalities and one additional “no finding” label).

• In our approach we have used an ensemble of fine-tuned(Transfer Learning) Xception architecture with DenseNet-121. The fine-tuning is done using state of the art approaches including cyclic learning rate.

• Transfer learning is when a model developed for one task is reused to work on a second task. Fine tuning is one approach to transfer learning.

​

​

Transfer learning

• Model weights can either be (i) randomly initialized, or (ii) transferred from a pre-trained model in a process called transfer learning. Transfer learning makes use of pre-trained weights. Pre-trained weights have useful information. We have used pretrained weights from imagenet dataset

• The usual model fitting for transfer learning works in the following way: train the weights that are closer to the output and freeze the other layers gradually.

• It is crucial that for transfer learning, one uses the same ‘stats’ that the pre-trained model was applied with, eg. correcting the image RGB values with a certain bias. To ensure that we have accordingly resized images to (224 x 224)

​

One Cycle Policy

• We have employed one cycle policy and cyclic momentum policy to fine tune our both networks.

• If the learning rate (LR) is too small, overfitting can occur. Large learning rates help to regularize the training but if the learning rate is too large, the training will diverge. Hence a grid search of shortruns to find learning rates that converge or diverge is possible but there is an easier way.

• Cyclical learning rates (CLR) and the learning rate range test (LR range test) were first proposed by Smith (2015) and later updated in Smith (2017) as a recipe for choosing the learning rate.

• Momentum and learning rate are closely related. The optimal learning rate is dependent on the momentum and momentum is dependent on the learning rate. Since learning rate is regarded as the most important hyper-parameter to tune (Bengio, 2012) then momentum is also important. Like learning rates, it is valuable to set momentum as large as possible without causing instabilities during training.

Conclusion

Our study will demonstrate that AI based Computer aided diagnosis – deep learning algorithm trained on a large quantity of well-labelled data can accurately detect multiple abnormalities on Chest X-rays. As these systems further increase in accuracy, the feasibility of applying deep learning to widen the reach of automatic chest X-ray interpretation and improving reporting efficiency will add tremendous value in radiology workflows and public health screenings in Indian healthcare sector, particularly rural India, which suffers from acute shortage of trained radiologists and doctors.

Future Direction

•We want to mitigate the interference of uncorrelated classes and preserve correlations among the relevant classes at the same time.

•To accomplish the above task we will be using combination of residual attention learning[4] and Spatial Regularization Net[5].
Our model will use a category-wise residual attention mechanism to assign different weights to different feature spatial regions.
It will automatically predict the attentive weights to enhance the relevant features and restrain the irrelevant features for a specific pathology.

Employing the above procedure should provide an improvement over existing approaches offered by Weng et. al. and Kapur et. al.

​

** References**

[1] Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R.; IEEE Conference on, 3462-3471 (2017)

[2] P. Rajpurkar et al., “CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning,” ArXiv171105225 Cs Stat, Nov. 2017

[3] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” ArXiv160806993 Cs, Aug. 2016.

[4] Fei Wang et. al. Residual Attention Network for Image Classification arXiv:1704.06904 CVPR 2017

[5] Feng Zhu et. al. Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification arXiv:1702.05891

Technologies Used

Python 3.7

PyTorch 1.2

CUDA 10, CuDNN 7.4

​

Repository

https://bitbucket.org/abhijit038/chest-x-ray-computer-aided-diagnosis-using-deep-learning/src/master/