The perception component of autonomous driving systems is often designed and tuned in isolation from the control component based on well-known performance measures such as accuracy, precision, and recall. Commonly used loss functions such as cross-entropy loss and negative log-likelihood only focus on minimizing the loss with respect to misclassification without considering the consequences that follow after the misclassifications. In other words, this approach fails to take into account the difference in the severity of system-level failures due to misclassification and other errors in perception components. Therefore in this work, we proposed a novel feedback learning training framework to build the perception component of an autonomous system that is aware of system-level safety objectives, which in turn, enhances the safety of the vehicle as a whole. The crux of the idea is to utilize the concept of a rulebook to provide feedback on system-level performance as safety scores and leverage them in designing and computing the loss functions for the models in the framework. The experimental results show the perception model gets improved by feedback from the system-level safety rule. The framework was trained and tested on an open-sourced dataset, and the experimental results showed that the resulting model had shown superior system-level safety performance over the baseline perception model.