Skip to yearly menu bar Skip to main content


Poster

Sharpness-Aware Minimization Activated Interactive Teaching Understanding and Optimization

MINGWEI XU · Xiaofeng Cao · Ivor Tsang

[ ]
Wed 11 Dec 11 a.m. PST — 2 p.m. PST

Abstract:

Teaching is a potentially effective approach for understanding interactions among multiple intelligences. Previous explorations have convincingly shown that teaching presents additional opportunities for observation and demonstration within the learning model, such as data distillation and selection. However, the underlying optimization principles and convergence of interactive teaching lack theoretical analysis. In this regard, co-teaching serves as a notable prototype. In this paper, we discuss its role as a reduction of the larger loss landscape derived from Sharpness-Aware Minimization (SAM). Then, we classify co-teaching as an iterative parameter estimation method using Expectation-Maximization (EM). The convergence of co-teaching is achieved by continuously optimizing a variational lower bound on the log marginal likelihood. This lower bound represents the expected value of the log posterior distribution of the latent variables under a scaled, factorized variational distribution. To further enhance interactive generalization performance, we incorporate SAM's strong generalization capabilities into co-teaching. This integration can be viewed as a novel sequential optimization problem. Finally, we validate the performance of our approach through multiple experiments.

Live content is unavailable. Log in and register to view live content