Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Bayesian Deep Learning

Stochastic Pruning: Fine-Tuning, and PAC-Bayes bound optimization

Soufiane Hayou · Bobby He · Gintare Karolina Dziugaite


Abstract:

We introduce an algorithmic framework for stochastic fine-tuning of pruning masks, starting from masks produced by several baselines. We further show that by minimizing a PAC-Bayes bound with data-dependent priors, we obtain a self-bounded learning algorithm with numerically tight bounds. In the linear model, we show that a PAC-Bayes generalization error bound is controlled by the magnitude of the change in feature alignment between the prior'' andposterior'' data.

Chat is not available.