ELLIS against Covid-19

CT analysis using machine learning to address COVID-19

ellis 15 April 2020 - 15 April 2020
15 April 2020 • 14:10 - 14:25

(Radboud University)

Chest computed tomography (CT) has been shown to have a key role in the diagnosis and management of COVID-19 and is increasingly used in symptomatic patients with suspected nCoV infection. Artificial Intelligence (AI) using deep learning has been advocated for automated reading of COVID-19 CT scans but how these automatically generated readings produce clinically actionable findings is not properly addressed. In this talk I would like to discuss how we can steer this effort towards clinically meaningful approaches and present the initiative we are currently carrying out in the Netherlands.


Questions & Answers

Link to the recording of the live Questions & Discussion session for this talk. 

  • Q: Do you have problems with domain shift when you apply your image analysis algorithms to hospitals that did not contribute training data?

    • A: We have applied the algorithms to images from different hospitals in the Netherlands and Germany. The initial results show that the performance is not affected but a proper validation has not been performed yet.

  • Q: The CO-RADS score levels 1 to 5 seem to be related by disease intensity. Levels 0 and 6 on the other side do not blend in as smoothly - this type of non-linearity could be a challenge and needs to be considered when looking for a ML approach that predicts the CO-RADS scores.

    • A: No, the scores are not related by disease intensity. CO-RADS provides a level of suspicion for pulmonary involvement of COVID-19 based on the features seen on CT. So they don't follow a linear scale. We take this into account for automatically predicting the score

  • Q: What is the inter-rater reliability for the CO-RAD score?

    • A: Agreements of individual observers with the median of the remaining observers were either substantial or moderate based on kappa values (more details in the CO-RADS paper that is currently accepted subject to minor revision)

  • Q: Did you need to retrain your lobe segmentation networks on the COVID-19 CT scans?

    • A: Yes, the system was initially pretrained on a cohort of 4000 subjects from the COPDGene study and later retrained on a small set of scans from COVID-19 patients

  • Q: We have seen recent works on differentiating between COVID-19 and Community-Acquired Pneumonia (CAP), what is your take on this?

    • A: Yes, I have. Some of the features of COVID-19 on CT are shared with other (viral) pneumonias, so distinguishing between them is a very hard task, even difficult for radiologists. This is reflected in the CORADS reporting scheme, producing an output that is clinically actionable. One drawback of some of the already published work is how the training/validation and test sets were created. The authors pull together different data sets of CT scans, one from a pneumonia study and another from a control study or for lung cancer study and another for covid. These scans might have been taken following a different protocol and, definitely, targeting a different population. Then I doubt if output of the algorithm is just based on these difference than on the underlying pathology.

  • Q: Could you please comment the power of the algorithm to separate between suspicious COVID and non-COVID cases and how this will compare to the usage of more traditional diagnostic tests?

    • A: The algorithm currently reaches a 0.92 area under the ROC curve for that task. In the paper where the CORADS score is evaluated (about to be published) is shown that CO-RADS was comparable to RT-PCR and clinical diagnosis of Covid-19 with an area of the ROC curve between 0.92 and 0.97.

  • Q: Ideally, how soon can the disease be detected with imaging? for CXR and CT? Can it really be used for diagnosis? Considering the speed at which the disease advances in subjects who suffer from it and taking into account the necessary radiation factor using CT/CXR is it really possible to measure the progression of the disease?

    • A: As I indicated in the talk, imaging is not recommended for asymptomatic patients so it is difficult to assess how early we can see features in the scans before symptoms appear. However, in some of our patients you could see COVID features before the PCR turned out positive. The power of CT is not only for its diagnosis value. With CT you can assess patients at risk for severe outcomes and the severity of the disease can be quantified. The scans are taking with a very low dose protocol adapted for these patients and it is being used for monitoring. I cannot comment further about the feasibility or added value as I'm not a clinician.

  • Q: How do you make sure that what seems a new scheme of thinking for radiologists with the CORADS score, does not bias their original way of (expert domain) thinking when rating these images for labelling data? i.e. is there anything the score cannot capture and the expert could that could be out of control?

    • A: Radiologists routinely use scoring schemes for their daily work (BI-RADS, LungRADS, PI-RADS) so they are used to this way of reporting.

  • Q: We have been talking to many radiologists and all agree that detecting COVID-19 with X-rays is nearly impossible, how AI could help here?

    • A: It is correct that it is a harder task. However, CXR are being used for triage (selecting patients that require an additional test for confirmation). In places where resources are not available, CXR has a significant added value. Additionally, we have seen that combining CXR with other clinical information a higher performance was obtained

  • Q: How much are the patterns found for Covid in image compared to other similar diseases and especially how reliable is this differentiation using AI?

    • A: Some of the findings seen in CT are typical for COVID and others are shared with other (viral) pneumonia. This is reflected in the CORADS reporting scheme and this is the output of our AI solution.

  • Q: Wouldn't it be reasonable to extend CT scanning with brain scans as well. For better understanding of neurological phenomenons. I understand that there's a cost of that, not only financial, though wouldn't it be reasonable to have a bigger picture?

    • A: I cannot comment about this. We are focused on routine clinical practice and providing solutions to facilitate and speed up decision making.


Thumb ticker clarisa s%c3%a1nchez
(University of Amsterdam)