Deep learning has made remarkable progress in recent years. These advances have been mostly in problems of unstructured data such as natural language processing, and computer vision. In contrast, most machine learning problems involve highly structured tabular data. This is true in many industries, including the financial industry. For tabular data, tree based machine learning methods, such as XGBoost are widely perceived as state of the art. Do the recent advances in deep learning have the opportunity to surpass the capabilities of tree based methods? We will review recent research on this topic, share what are some learnings from the various ideas that have been tried out, and point out what are the things that deep learning methods will be able to do that tree based methods do not provide us.