API aggregateΒΆ

The aggregate strategies are used to combine labels from multiple users into a single label. This aggregated label can be either a probability distribution (soft label) or a single class label (hard label).

All strategies are located at this direction on GitHub <https://github.com/peerannot/peerannot/tree/main/peerannot/models/aggregation>.

All aggregation-based strategies are available running:

peerannot agginfo

MV.MV

Majority voting Most answered label per task

NaiveSoft.NaiveSoft

Naive soft: Frequency distribution

Wawa.Wawa

Worker Agreement With Aggregate

IWMV.IWMV

Iterative Weighted Majority Vote

DS.Dawid_Skene

Dawid and skene model (1979)

GLAD.GLAD

GLAD (Whitehill et. al 2009)

plantnet.PlantNet

PlantNet aggregation strategy

twothird.TwoThird

Two third agreement Accepts the label given with a two third consensus on at least 2 votes and returns -1 otherwise

DS_clust.Dawid_Skene_clust

WDS.WDS

WDS: Weighted Distribution from Dawid and Skene