Contact-rich manipulation remains a major challenge in robotics. Optical tactile sensors like GelSight Mini offer a low-cost solution for contact sensing by capturing soft-body deformations of the silicone gel. However, accurately inferring shear and normal force distributions from these deformations has yet to be fully addressed. In this work, we propose a machine learning approach using a U-net architecture to predict force distributions directly from raw images. Our model, trained on force distributions inferred from FEA, demonstrates promising accuracy in predicting these distributions. It also shows potential for generalizing to a different GelSight sensor than the one used for data collection and enables real-time application. This approach holds the promise of improving tactile perception in contact-rich robotic manipulation tasks.