In the field of natural language generation (NLG), evaluating the interestingness and creativity of automatically generated stories remains a significant challenge. Traditional metrics focus on structural coherence, character development, and plot complexity, and there is an increasing demand for metrics that assess story interestingness and creativity. To address this gap, we introduce ICScore (Interestingness and Creativity Score), a novel metric for assessing story engagement using pre-trained language models (PLMs). ICScore is derived by calculating the likelihood differences between original and modified stories, incorporating six key factors of interestingness: metaphor and simile, irony, pun, engaging plot, vivid writing, and sensory description. Our results demonstrate a strong correlation between ICScore and human evaluations, validating the metric's effectiveness. ICScore is model independent and requires no specific training data, enhancing its applicability across diverse story generation systems. This metric offers a more objective and efficient alternative to human evaluation, providing valuable insights for the development of engaging and creative narratives. Our code is available on a GitHub repository at link https://anonymous.4open.science/r/ICScore-DCB9