Prediction of average voltage of a cathode material, which is related to energy density, is an important task in a battery. However, it is difficult to develop a practical prediction model because relevant data is small, and important information including structure, regarded as a good modality for predicting properties of materials, is barely known except compositions. Inspired by these points, we propose a pretraining method utilizing a contrastive learning between compositions and structures(CLCS), which can improve the performance of voltage prediction task using only compositions of materials. First, we pretrained an composition encoder through contrastive learning between composition and structure representations, extracted by a transformer encoder and a graph neural network respectively, enabling the composition encoder to learn information associated with structures. Then, we transferred the composition encoder to a downstream task of predicting the average voltage with compositions. The performance of transferred model exceeds one of a model without pretraining by 9.7%. Also, with attention score analysis, we discovered that the transferred composition encoder focuses on lithium more than other elements in lithium-transition metal-oxygen systems compared to the composition encoder without pretraining.