Foundation Industries (FIs), constitute glass, metals, cement, ceramics, bulk chemicals, paper, steel, etc, and provide crucial, foundational, materials for a diverse set of economically relevant industries: automobiles, machinery, construction, household appliances, chemicals, etc. Reheating furnaces within the manufacturing chain of FIs are energy-intensive. Deep Learning (DL) powered control systems could lead to notable energy consumption reduction by reducing the overall heating time in furnaces. This could help achieve the Net-Zero goals in FIs for sustainable manufacturing. In this work, due to the infeasibility of achieving good quality data in scenarios like reheating furnaces, classical Hottel's zone method based computational model has been used to generate data for DL based model training via regression. To further enhance the Out-Of-Distribution (OOD) generalization capability of the trained DL model, we propose a Physics-Informed Neural Network (PINN) by incorporating prior physical knowledge using a set of novel Energy-Balance regularizers.