Preserving AUC Fairness in Learning with Noisy Protected Groups

Mingyang Wu, Li Lin, Wenbin Zhang, Xin Wang, Zhenhuan Yang, Shu Hu

This repository is the official implementation of our paper "Preserving AUC Fairness in Learning with Noisy Protected Groups", which has been accepted by ICML 2025.

1. Installation

You can set up the environment using the following commands:

cd AUC-Fairness-Noisy
conda create -n FairnessNoisy python=3.9.0
conda activate FairnessNoisy
pip install -r requirements.txt

2. Dataset Preparation

We share the tabular dataset Adult, Bank, and Default datasets with demographic annotations from paper, and image datasets FF++, Celeb-DF, DFD, and DFDC datasets with demographic annotations from paper, which can be downloaded through this link.

You can also access these re-annotated image datasets with prediction uncertainty scores via our AI-Face-FairnessBench.

Alternatively, you can download these datasets from their official sources and process them by following these steps:

• Download FF++, Celeb-DF, DFD, and DFDC datasets.
• Download annotations for these datasets according to this paper and their code, then extract the demographic information from all images.
• Extract, align, and crop faces using DLib, and save them to:

  /path/to/cropped_images/

• **Split cropped images into train/val/test with a ratio of 60%/20%/20%, ensuring no identity overlap.

3. Load Pretrained Weights

Before running the training code, make sure you load the pre-trained weights. We provide pre-trained weights under
pretrained. You can also download the Xception model trained on ImageNet (through this link) or use your own pretrained Xception.

---

4. Train

To run the training code, first navigate to the ./training/ folder, then execute the following command to train our detector with loss flattening strategy: For image datasets:

cd training
python train_images.py
--lr: learning rate, default is 0.0005.
--train_batchsize: batch size, default is 32.
--seed: random seeds
--datapath: /path/to/train.csv
--model: detector name: auc_effinet
--device: gpu ids for training.
--num_groups: default is 4
--method: ours is 'auc'
--backbone: default is 'efficientnetb4'
--pho: learning parameter for SAM, default is 0.5
--ratio: noise ratio, default is 0.02

For tabular datasets:

cd training
python train_tabular.py
--lr: learning rate, default is 0.01.
--train_batchsize: batch size, default is 10000.
--seed: random seeds
--device: gpu ids for training.
--pho: learning parameter for SAM, default is 0.0005
--method: ours is 'auc'
--dataset: training dataset name: default.
--ratio: noise ratio, default is 0.02

5. Test

For model testing, we provide a python file to test our model by running python test.py.

--checkpoints : /path/to/saved/model.pth
--model: 'auc_effinet'
--device: gpu ids for training.
--inter_attribute : intersectional group names divided by '-': male-female
--label_attribute : label attribute name divided by '-': pos-neg
--test_datapath : /path/to/test.csv
--savepath : /where/to/save/predictions.npy(labels.npy)/results/
--model_structure : backbone: auc_effinet.
--test_batch_size : testing batch size: default is 32.

Citation

Please kindly consider citing our papers in your publications.

@inproceedings{wu2025preserving,
  title={Preserving auc fairness in learning with noisy protected groups},
  author={Wu, M and Lin, L and Zhang, W and Wang, X and Yang, Z and Hu, S},
  booktitle={The 42nd International Conference on Machine Learning (ICML)},
  year={2025}
}