Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Machine Learning with New Compute Paradigms

Adjoint Method: The Connection between Analog-based Equilibrium Propagation Architectures and Neural ODEs

Mohamed Watfa · Alberto Garcia-Ortiz


Abstract:

Analog neural networks (ANNs) hold significant potential for substantialreductions in power consumption in modern neural networks, particularly whenemploying the increasingly popular Energy-Based Models (EBMs) in tandem withthe local Equilibrium Propagation (EP) training algorithm. This paper analyzesthe relationship between this family of ANNs and the concept of Neural OrdinaryDifferential Equations (Neural ODEs). Using the adjoint method, we formallydemonstrate that ANN-EP can be derived from Neural ODEs by constraining thedifferential equations to those with a steady-state response. This findingopens avenues for the ANN-EP community to extend ANNs to non-steady-statescenarios. Additionally, it provides an efficient setting for NN-ODEs thatsignificantly reduces the training cost.

Chat is not available.