Coastal zones are under increasing pressures due to climate change and the increasing population densities in coastal areas around the globe. Our ability to accurately forecast the evolution of the coastal zone is of critical importance to coastal managers in the context of risk assessment and mitigation. Recent advances in artificial intelligence and remote sensing enable the development of automatic large-scale analysis methodologies based on observation data. In this work, we make use of a novel satellite-derived shoreline forecasting dataset and a variant of the common Encoder-Decoder neural network, UNet, in order to predict shoreline change based on spatio-temporal data. We analyze the importance of including the spatial context at the prediction step and we find that it greatly enhances model performance. Overall, the model presented here demonstrates significant shoreline forecasting skill around the globe, achieving a global correlation of 0.77. The code, data, and trained models are available online at https://anonymous.4open.science/r/DL-spatiotemporal-shorelines.