Poster Session
in
Workshop: Scientific Methods for Understanding Neural Networks
Is Saliency Really Captured By Gradient?
Nehal Yasin · Jonathon Hare · Antonia Marcu
Abstract:
Numerous feature attribution (or saliency) measures have been proposed that utilise the gradients of the output with respect to features. Gradients in this setting unequivocally tell us about feature sensitivity by definition of the gradient, but do they really tell us about feature importance? We challenge the idea that sensitivity and importance are the same, and empirically show that gradients do not necessarily find important features that should be attributed to a models' prediction.
Chat is not available.