【Data】 Domain Generalization and Long-tailed dataset

02 Dec 2022 in Artificial Intelligence

Domain Generalization and Long-tailed dataset

Long-tailed distribution

• (Train) Unbalanced train dataset -> (Test) Balanced test dataset
• Early Methods
  1. Re-sampling / Synthesizing
     • Feature Space Augmentation for Long-Tailed Data, ECCV20
       • Using CAM, the class-specific features and class-generic features are divided.
     • The Majority Can Help The Minority, CVPR22
       • Using CutMix
       • Background -> the majority class // Foreground -> the minority class
       • Simple but effective
  2. Re-weighting
     • Focal loss, Class-balanced loss
     • Label distribution-aware Margin Loss, NIPS 2019
       • Handle decision boundary.
       • Utilize ‘Normed Linear’
       • the small number of classes -> large margin in the learning rate annealing stage.
  3. Two-stage (Backbone VS Classifier)
     • Decoupling Representation and Classifier, ICLR20
       • Train an entire model using class-balanced loss.
       • Train a fully connected model using output normalization.
     • Weight Balancing, CVPR22
       • Weight decay and MaxNorm
       • Simple but impactful
  4. Representation
     • Supervised Contrastive learning, CVPR21
       • Two classifiers (1) cross entropy (2) supervised contrastive loss
     • Parametric Contrastive learning, ICCV21
       • Momentum encoder
       • Parametric learnable class centers (balanced cross-entropy loss)
  5. Logit Adjustment
     • Balanced Softmax, NIPS20
       • The balanced softmax adjusts the prediction probability.
     • Distribution Alignment, CVPR21
  6. Ensemble
     • Without performance loss In balanced data (False positive ↑), improve performance.
     • Bilateral-Branch Network, CVPR20
       • Resampling when training a classifier. (reverse distribution)
       • Cross entropy when training representation. (uniform distribution)
     • RIDE: Routing Diverse Experts, ICLR21
       • Multiple experts
       • Distribution-aware diversity loss
     • Self-supervised Aggregation of Diverse Experts, NIPS22

-