In recent years, AI for Social Good (AI4SG) projects have grown in scope and popularity, covering a variety of topics from climate change to education and being the subject of numerous workshops and conferences at a global scale. In the current article, we reflect upon AI4SG, its definition and its current limitations. We propose ways to address these limitations, from connecting with relevant disciplines to a better consideration of the constraints and context of project deployment. We conclude with a proposal to refocus the field of AI4SG around the concept of sustainability from a variety of angles, arguing that this will help the field evolve while taking its own impacts into account.