TwoThird

class TwoThird(answers, n_classes=2, sparse=False, **kwargs)

Two third agreement

Accepts the label given with a two third consensus on at least 2 votes and returns -1 otherwise

__init__(answers, n_classes=2, sparse=False, **kwargs)

Two Third agreement: accept label reaching two third consensus

\[\begin{split}\mathrm{TwoThird}(i, \{y_i^{(j)}\}_j) = \begin{cases} \mathrm{MV}(i, \{y_i^{(j)}\}_j) & \text{if} s_i=1 \\ \text{undefined} & \text{otherwise} \end{cases}\end{split}\]

Parameters:

• answers (dict) –

  Dictionary of workers answers with format

  {
      task0: {worker0: label, worker1: label},
      task1: {worker1: label}
  }
  

• n_classes (int, optional) – Number of possible classes, defaults to 2

• sparse (bool, optional) – If the number of workers/tasks/label is large (\(>10^{6}\) for at least one), use sparse=True to run per task

get_probas()

Two third strategy does not return soft labels. Defaults to get_answers()

Raises:

Warning – TwoThird agreement only returns hard labels, using get_answers()

get_answers()

Argmax of soft labels, in this case corresponds to a majority vote

with two third consensus. If the consensus is not reached, a -1 is used as input. Additionally, if a dataset path is provided, tasks index with a -1 label are saved at <dataset>/identification/twothird/too_hard.txt

CLI only: the <dataset> key is the shared input between aggregation strategies used as follows

peerannot aggregate <dataset> --answers answers.json -s <strategy>`

Returns:

Hard labels and None when no consensus is reached

Return type:

numpy.ndarray