Wawa

class Wawa(answers, n_classes=2, sparse=False, **kwargs)

Worker Agreement With Aggregate

__init__(answers, n_classes=2, sparse=False, **kwargs)

WAWA aggregation weighs each worker by their accuracy with a majority vote.

\[ \begin{align}\begin{aligned}\mathrm{WAWA}(i, \mathcal{D}) = \underset{k\in[K]}{\mathrm{argmax}} \sum_{j\in\mathcal{A}(x_i)}\beta_j \mathbf{1}(y_i^{(j)} = k)\\\beta_j = \frac{1}{|\{y_{i'}^{(j)}\}_{i'}|} \sum_{i'=1}^{n_{\texttt{task}}} \mathbb{1}(y_{i'}^{(j)} = \mathrm{MV}(i', \{y_{i'}^{(j)}\}_j)\end{aligned}\end{align} \]

Parameters:

• answers (dict) –

  Dictionary of workers answers with format

  {
      task0: {worker0: label, worker1: label},
      task1: {worker1: label}
  }
  

• sparse (bool, optional) – If the number of workers/tasks/label is large (\(>10^{6}\) for at least one), use sparse=True to run per task

• n_classes (int, optional) – Number of possible classes, defaults to 2

run(**kwargs)

Runs a single step of IWMV aggregation

get_answers()

Get labels obtained with majority voting aggregation

Returns:

Most answered labels per task

Return type:

numpy.ndarray

get_probas()

Get labels obtained with majority voting aggregation

Raises:

Warning – WAWA aggregation only returns hard labels, using get_answers()

Returns:

Most answered labels per task

Return type:

numpy.ndarray