PEFAT: Boosting semi-supervised medical image classification with pseudo-loss estimation and feature adversarial training

Qingjie Zeng, Yutong Xie, Zilin Lu, Yong Xia — Northwestern Polytechnical University and The University of Adelaide

2023

Semi-supervised medical image classification workflow using pseudo-loss estimation and feature adversarial training

Publication summary

This CVPR 2023 paper introduces PEFAT, a semi-supervised learning framework designed to improve medical image classification when only a small amount of labeled data is available. Instead of relying only on model confidence to select pseudo-labels, PEFAT estimates pseudo-label quality through loss distribution modeling.

The method first warms up the model with contrastive learning, then fits a two-component Gaussian Mixture Model on labeled-data losses. For unlabeled samples, cross pseudo-loss estimation is used to identify trustworthy pseudo-labeled examples, helping reduce the risk of confirmation bias from overconfident but incorrect predictions.

PEFAT also introduces Feature Adversarial Training, which injects adversarial noise at the feature level so that even unselected or uncertain unlabeled samples can still help smooth the decision boundary. Experiments across NCT-CRC-HE, ISIC2018, Chest X-Ray14, CIFAR-10, and CIFAR-100 show that PEFAT outperforms several advanced semi-supervised baselines.

“PEFAT shows that unlabeled clinical data should not simply be filtered by confidence. By using loss behavior and feature-level adversarial learning, the model can extract more value from limited annotations while reducing pseudo-label noise.”

Editorial research summary

OrthoAI research content adaptation

Key findings

98.25%

AUC on NCT-CRC-HE with only 100 labeled samples, outperforming FixMatch, SimPLE, CoMatch, and SimMatch.

94.87%

AUC on ISIC2018 using 20% labeled data, with the best reported results across all listed evaluation metrics.

82.58%

Mean AUC on Chest X-Ray14 with 20% labeled data, exceeding prior semi-supervised medical image classification methods.

Method

Cross pseudo-loss estimation

PEFAT replaces simple probability-threshold pseudo-label selection with a loss-distribution strategy. After contrastive warm-up, the model calculates losses on labeled samples and fits a two-component Gaussian Mixture Model to separate likely clean and noisy pseudo-label behavior.

For unlabeled images, two strong augmented views are cross-predicted against each other. The resulting cross pseudo-losses are passed into the fitted GMM, allowing the framework to select pseudo-labeled samples based on estimated trustworthiness rather than raw confidence.

Pseudo-loss estimation pipeline for selecting high-quality pseudo-labeled medical images

Feature adversarial training

Instead of discarding lower-confidence unlabeled data, PEFAT uses feature-level adversarial perturbations to learn from those samples. The adversarial noise is injected into feature embeddings, encouraging consistent predictions and a smoother decision boundary.

The paper shows that this feature-level approach creates more separable representations than image-level virtual adversarial training, particularly for samples near decision boundaries.

Feature adversarial training for semi-supervised medical image classification

Key contributions

PEFAT addresses two core limitations in semi-supervised medical image classification: unreliable pseudo-label selection and underuse of uncertain unlabeled data.

Introduces pseudo-loss estimation as an alternative to probability-threshold pseudo-label selection.
Uses a two-component Gaussian Mixture Model to estimate whether pseudo-labeled samples are likely clean or noisy.
Applies cross pseudo-loss estimation across augmented views to reduce confirmation bias.
Introduces Feature Adversarial Training for extracting useful information from unselected unlabeled samples.
Improves decision-boundary smoothness through feature-level perturbation rather than only image-level augmentation.
Demonstrates strong results on three public medical datasets and two natural image datasets.

Similar publications

CVPR 2024 · IEEE/CVF Conference on Computer Vision and Pattern Recognition

PairAug: What Can Augmented Image-Text Pairs Do for Radiology?

Yutong Xie, Qi Chen, Sinuo Wang, Minh-Son To, Iris Lee, Ee Win Khoo, Kerolos Hendy, Daniel Koh, Yong Xia, Qi Wu

view

CVPR 2024 · IEEE/CVF Conference on Computer Vision and Pattern Recognition

Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning

Yiwen Ye, Yutong Xie, Jianpeng Zhang, Ziyang Chen, Qi Wu, Yong Xia

view

view all