Regularization is a key technique to avoid overfitting and to improve generalization of deep learning models. In many scientific and engineering applications, an approximate model of the complex system is usually known, although with both aleatoric and epistemic uncertainties. We present a principled method to incorporate these approximate models as physics priors in model training, by structuring the priors as generalized regularizers. The experimental results demonstrate that our method achieves one to two orders of magnitude of improvement in testing accuracy