Human not in the loop: objective sample difficulty measures for Curriculum Learning

Zhengbo Zhou¹, Jun Luo¹, Dooman Arefan¹, Gene Kitamura², Shandong Wu¹,

¹University of Pittsburgh, ²Loma Linda University

ISBI 2023

arXiv

Curriculum learning based on Variance of Gradient

Abstract

Curriculum learning is a learning method that trains models in a meaningful order from easier to harder samples. A key here is to devise automatic and objective difficulty measures of samples. In the medical domain, previous work applied domain knowledge from human experts to qualitatively assess classification difficulty of medical images to guide curriculum learning, which requires extra annotation efforts, relies on subjective human experience, and may introduce bias. In this work, we propose a new automated curriculum learning technique using the variance of gradients (VoG) to compute an objective difficulty measure of samples and evaluated its effects on elbow fracture classification from X-ray images. Specifically, we used VoG as a metric to rank each sample in terms of the classification difficulty, where high VoG scores indicate more difficult cases for classification, to guide the curriculum training process We compared the proposed technique to a baseline (without curriculum learning), a previous method that used human annotations on classification difficulty, and anti-curriculum learning. Our experiment results showed comparable and higher performance for the binary and multi-class bone fracture classification tasks.

BibTeX

@inproceedings{zhou2023human,
      title={Human not in the loop: Objective sample difficulty measures for curriculum learning},
      author={Zhou, Zhengbo and Luo, Jun and Arefan, Dooman and Kitamura, Gene and Wu, Shandong},
      booktitle={2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI)},
      pages={1--5},
      year={2023},
      organization={IEEE}
    }
  

Deep Curriculum Learning in Task Space for Multi-Class Based Mammography Diagnosis

Jun Luo¹, Dooman Arefan¹, Margarita Zuley¹, Jules Sumkin¹, Shandong Wu¹,

¹University of Pittsburgh,

SPIE Medical Imaging 2022

arXiv Slides

Task-space curriculum loss scheduler

Abstract

Mammography is used as a standard screening procedure for the potential patients of breast cancer. Over the past decade, it has been shown that deep learning techniques have succeeded in reaching near-human performance in a number of tasks, and its application in mammography is one of the topics that medical researchers most concentrate on. In this work, we propose an end-to-end Curriculum Learning (CL) strategy in task space for classifying the three categories of Full-Field Digital Mammography (FFDM), namely Malignant, Negative, and False recall. Specifically, our method treats this three-class classification as a "harder" task in terms of CL, and create an "easier" sub-task of classifying False recall against the combined group of Negative and Malignant. We introduce a loss scheduler to dynamically weight the contribution of the losses from the two tasks throughout the entire training process. We conduct experiments on an FFDM datasets of 1,709 images using 5-fold cross validation. The results show that our curriculum learning strategy can boost the performance for classifying the three categories of FFDM compared to the baseline strategies for model training.

BibTeX

@inproceedings{luo2022deep,
      title={Deep curriculum learning in task space for multi-class based mammography diagnosis},
      author={Luo, Jun and Arefan, Dooman and Zuley, Margarita and Sumkin, Jules and Wu, Shandong},
      booktitle={Medical Imaging 2022: Computer-Aided Diagnosis},
      volume={12033},
      pages={85--90},
      year={2022},
      organization={SPIE}
    }
  

Knowledge-Guided Multiview Deep Curriculum Learning for Elbow Fracture Classification

Jun Luo¹, Gene Kitamura², Dooman Arefan¹, Emine Doganay¹, Ashok Panigrahy¹, Shandong Wu¹,

¹University of Pittsburgh, ²Loma Linda University

MICCAI-W 2021

arXiv Code Poster Slides

Workflow of multiview curriculum learning

Abstract

Elbow fracture diagnosis often requires patients to take both frontal and lateral views of elbow X-ray radiographs. In this paper, we propose a multiview deep learning method for an elbow fracture subtype classification task. Our strategy leverages transfer learning by first training two single-view models, one for frontal view and the other for lateral view, and then transferring the weights to the corresponding layers in the proposed multiview network architecture. Meanwhile, quantitative medical knowledge was integrated into the training process through a curriculum learning framework, which enables the model to first learn from "easier" samples and then transition to "harder" samples to reach better performance. In addition, our multiview network can work both in a dual-view setting and with a single view as input. We evaluate our method through extensive experiments on a classification task of elbow fracture with a dataset of 1,964 images. Results show that our method outperforms two related methods on bone fracture study in multiple settings, and our technique is able to boost the performance of the compared methods. The code is available at https://github.com/ljaiverson/multiview-curriculum.

BibTeX

@inproceedings{luo2021knowledge,
        title={Knowledge-guided multiview deep curriculum learning for elbow fracture classification},
        author={Luo, Jun and Kitamura, Gene and Arefan, Dooman and Doganay, Emine and Panigrahy, Ashok and Wu, Shandong},
        booktitle={Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 12},
        pages={555--564},
        year={2021},
        organization={Springer}
      }
    

Medical knowledge-Guided Deep Curriculum Learning for Elbow Fracture Diagnosis from X-Ray Images

Jun Luo¹, Gene Kitamura², Emine Doganay¹, Dooman Arefan¹, Shandong Wu¹,

¹University of Pittsburgh, ²Loma Linda University

SPIE Medical Imaging 2021

PDF Slides

Knowledge-guided currciulum mechanism

Abstract

Elbow fractures are one of the most common fracture types. Diagnoses on elbow fractures often need the help of radiographic imaging to be read and analyzed by a specialized radiologist with years of training. Thanks to the recent advances of deep learning, a model that can classify and detect different types of bone fractures needs only hours of training and has shown promising results. However, most existing deep learning models are purely data-driven, lacking incorporation of known domain knowledge from human experts. In this work, we propose a novel deep learning method to diagnose elbow fracture from elbow X-ray images by integrating domain-specific medical knowledge into a curriculum learning framework. In our method, the training data are permutated by sampling without replacement at the beginning of each training epoch. The sampling probability of each training sample is guided by a scoring criterion constructed based on clinically known knowledge from human experts, where the scoring indicates the diagnosis difficultness of different elbow fracture subtypes. We also propose an algorithm that updates the sampling probabilities at each epoch, which is applicable to other sampling-based curriculum learning frameworks. We design an experiment with 1865 elbow X-ray images for a fracture/normal binary classification task and compare our proposed method to a baseline method and a previous method using multiple metrics. Our results show that the proposed method achieves the highest classification performance. Also, our proposed probability update algorithm boosts the performance of the previous method.

BibTeX

@inproceedings{luo2021medical,
        title={Medical knowledge-guided deep curriculum learning for elbow fracture diagnosis from x-ray images},
        author={Luo, Jun and Kitamura, Gene and Doganay, Emine and Arefan, Dooman and Wu, Shandong},
        booktitle={Medical Imaging 2021: Computer-Aided Diagnosis},
        volume={11597},
        pages={247--252},
        year={2021},
        organization={SPIE}
      }