Naive Soft

Model

The Naive Soft model returns the empirical distribution overs answered for each task:

\[\DeclareMathOperator*{\argmax}{argmax} \hat y_i = \big( \frac{1}{|n_\texttt{worker|}}\sum_{j\in [n_\texttt{worker}]} \unicode{x1D7D9}_{\{y_i^{(j)}=k\}}\big).\]

CLI

With peerannot in a terminal located in the directory of answers, the DS model can be used as follows.

peerannnot aggregate . --strategy NaiveSoft --answers-file answers.json

Note that by default, if the answers are in a file names answers.json the --answers-file argument can be omitted.

API

Import the aggregation model in the current session

from peerannot.models import NaiveSoft as NS

Assuming the answers are in a dictionary names answers then run:

ns = NS(answers, n_classes)
yhat = ns.get_probas()

Note that the Naive Soft aggregation produces labels in the simplex by default.

API details: class models.NaiveSoft

NS model class inherits from CrowdModel

__init__(answers, n_classes,**kwargs)

Parameters:

  • answers:(dict) Dictionnary of workers answers with format 
    
          {
              task0: {worker0: label, worker1: label},
              task1: {worker1: label}
          }
  • n_classes: (int) Number of possible classes
  • kwargs: (dict) Dictionary that can contain path_remove to remove tasks identified from the WAUM or another method.

compute_baseline()

For each given task, computes the number of votes for each label.

get_probas()

Returns the distribution of voted classes from the baseline.

get_answers()

Returns the most voted class from the baseline. In case of equalities, a random choice is applied.