Geographic location is essential for modeling tasks in climate-related fields ranging from ecology to the Earth system sciences. Here, a meaningful feature representation of locations is highly helpful as a description that encodes location-specific aspects. However, obtaining such a representation is challenging and requires an algorithm to distill semantic information of one location from available data. To address this challenge, we introduce GeoCLIP, a global, general-purpose geographic location encoder that provides vector embeddings summarizing the characteristics of a given location for convenient usage in diverse downstream tasks. We show that GeoCLIP embeddings, pretrained on multi-spectral Sentinel-2 satellite data, can be used for various predictive out-of-domain tasks, including temperature prediction and animal recognition in imagery, and outperform existing competing approaches. This demonstrates the potential of general-purpose location encoders and opens the door to learning meaningful representations of our planet from the vast, varied, and largely untapped modalities of geospatial data.