Poster
On Margins and Generalisation for Voting Classifiers
Felix Biggs · Valentina Zantedeschi · Benjamin Guedj
Hall J (level 1) #935
Keywords: [ ensemble learning ] [ Margins ] [ Generalisation bounds ] [ Majority votes ] [ Aggregation of experts ] [ PAC-Bayes ]
We study the generalisation properties of majority voting on finite ensembles of classifiers, proving margin-based generalisation bounds via the PAC-Bayes theory. These provide state-of-the-art guarantees on a number of classification tasks. Our central results leverage the Dirichlet posteriors studied recently by Zantedeschi et al. (2021) for training voting classifiers; in contrast to that work our bounds apply to non-randomised votes via the use of margins. Our contributions add perspective to the debate on the ``margins theory'' proposed by Schapire et al. (1998) for the generalisation of ensemble classifiers.