CLI identify

The help documentation is available in the terminal from:

peerannot identify --help

We created an interactive tool to compare the WAUM and AUM identifications on the CIFAR-10H dataset. Check it out by clicking this link!

peerannot identify

Identify ambiguous tasks using different methods available in peerannot identificationinfo

peerannot identify [OPTIONS] [FOLDERPATH]

Options

--hard-labels <hard_labels>

Path to file of hard labels (only for AUM)

-K, --n-classes <n_classes>

Number of classes

-s, --method <method>

Method to find ambiguous tasks

--labels <labels>

Path to file of crowdsourced answers

--use-pleiss

Use Pleiss et. al (2020) margin instead of Yang’s

Default:

False

--model <model>

Name of neural network to use. The list is available at peerannot modelinfo

--n-epochs <n_epochs>

Number of training epochs

--alpha <alpha>

Cutoff hyperparameter

--topk <topk>

Use TopK WAUM with k=XXX

--n-params <n_params>

Number of parameters for the logistic regression only

--lr <lr>

Learning rate

--pretrained

Use torch available weights to initialize the network

Default:

False

--momentum <momentum>

Momentum for the optimizer

--metadata_path <metadata_path>

Path to the metadata of the dataset if different than default

--decay <decay>

Weight decay for the optimizer

--img-size <img_size>

Size of image (square)

--maxiter-DS <maxiter_ds>

Maximum number of iterations for the Dawid and Skene algorithm

-optim, --optimizer <optimizer>

Optimizer for the neural network

--data-augmentation

Perform data augmentation on training set with a random choice between RandomAffine(shear=15), RandomHorizontalFlip(0.5) and RandomResizedCrop

Default:

False

--freeze

Freeze all layers of the network except for the last one

Default:

False

--matrix-file <matrix_file>

Path to confusion matrices saved with an aggregation method like DS. If not provided, run DS model

--hard-labels <hard_labels>

Path to file of hard labels

--seed <seed>

random seed

Arguments

FOLDERPATH

Optional argument