Poster
in
Workshop: Bayesian Deep Learning
Stochastic Pruning: Fine-Tuning, and PAC-Bayes bound optimization
Soufiane Hayou · Bobby He · Gintare Karolina Dziugaite
Abstract:
We introduce an algorithmic framework for stochastic fine-tuning of pruning masks, starting from masks produced by several baselines. We further show that by minimizing a PAC-Bayes bound with data-dependent priors, we obtain a self-bounded learning algorithm with numerically tight bounds. In the linear model, we show that a PAC-Bayes generalization error bound is controlled by the magnitude of the change in feature alignment between the prior'' and
posterior'' data.
Chat is not available.