Significant progress has been made to obtain approximate solutions to PDEs using neural networks as a basis. One of these approaches (and the most popular and well-developed one) is the Physics Informed Neural Network (PINN). PINN has proved to provide promising results in various forward and inverse problems with great accuracy. However, PINN cannot be employed in its native form for solving problems where the PDE changes its form or when there is a discontinuity in the parameters of PDE across different sub-domains. Using separate PINNs for each sub-domain and connecting the corresponding solutions by interface conditions is a possible solution for this. However, this approach demands a high computational burden and memory usage. Here, we present a new method, Transfer Physics Informed Neural Network (TPINN), where one or more layer of PINN across different non overlapping sub-domains are changed keeping the other layers same for all the sub-domains. Solutions from different sub-domains are connected via problem specific interface conditions which are incorporated in to the loss function. We demonstrate the efficacy of TPINN through two heat transfer problems.