CoNAL

class CoNAL(tasks_path, answers, model, n_classes, optimizer, n_epochs, scale=1e-05, verbose=True, pretrained=False, output_name='conal', **kwargs)

CoNAL (Common Noise Adaptation Layer), Chu et.al 2021

Implementation based from the unofficial repository https://github.com/seunghyukcho/CoNAL-pytorch

__init__(tasks_path, answers, model, n_classes, optimizer, n_epochs, scale=1e-05, verbose=True, pretrained=False, output_name='conal', **kwargs)

CoNAL deep learning strategy. Learn a classifier with crowdsourced labels by modeling worker-specific and global confusions.

During training, given the classifier \(\mathcal{C}\) with output scores \(z_i\), local confusions \(\pi^{(j)}\) and global confusion \(\pi^g\), the model computes the following:

\[h_i^{(j)} = \sigma((\omega_{i}^{(j)}\pi^g + (1-\omega_i^{(j)})\pi^{(j)} ) z_i ),\]

with \(\omega_i^{(j)}=(1+\exp(-u_j^\top v_i))^{-1}\) where \(u_j\) is the worker-related embedding and \(v_i\) the task-related embedding. This is computed in the AuxiliaryNetwork. Then, the NoiseAdaptationLayer computes the final prediction \(h_i^{(j)}\).

The final loss is the crossentropy between the worker-specific prediction and the given label with a regularization term of penalty scale \(\lambda>0\) on the difference between the local and global confusions.

\[\mathcal{L} = \frac{1}{n_{\texttt{task}}}\sum_{i=1}^{n_{\texttt{task}}}\sum_{j\in\mathcal{A}(x_i)} \mathrm{CE}(h_i^{(j)}, y_i^{(j)}) - \lambda \sum_{j=1}^{n_{\texttt{worker}}} \|\pi^{(j)} - \pi^g\|_2.\]

Parameters:

• tasks_path (path) – Path to images to train from

• answers (path) –

  Path to answers (json format)

  {
      task0: {worker0: label, worker1: label},
      task1: {worker1: label}
  }
  

• model (string) – Backbone classifier architecture: should be from torchvision.models

• n_classes (int) – Number of classes

• optimizer (string) – Pytorch optimizer name (either sgd or Adam)

• n_epochs (int) – Number of training epochs

• scale (float) – Regularization parameter in the loss, defaults to 1e-5

• verbose (bool, optional) – Verbosity level, defaults to True

• pretrained (bool, optional) – Use the pretrained version of the backbone classifier, defaults to False

• output_name (str, optional) – Generated file prefix, defaults to “conal”

The batch size, learning rate, scheduler and milestones can be specified as keyword arguments. Visit the computo paper or the tutorial for examples.

setup(**kwargs)

Create training, validation and test dataloaders from the dataset.

run(**kwargs)

Train the CoNAL model and evaluate on the test set.

Uses a gpu by default is torch.cuda.is_available() is True.

Results are stored in <tasks_path>/results/<output_name>.json and the best model in <tasks_path>/best_models/<output_name>.pth. Results contain the train, validation and test loss as well as the validation and test accuracy.

run_epoch(model, trainloader, criterion, optimizer, logger)

Run one epoch and monitor metrics