Digital images in real-world applications mainly suffer from several quality degradations. Most learning-based codecs rely on a predefined compression process with perfect or high-quality images as the input. Nevertheless, compared to high-quality photos, images in the wild present very different features, making learning-based image coding sub-optimal. This paper offers a framework for compressing distorted images while estimating distortion-free ones. The reconstructed signals are represented using two categories of latent space features: the features that illustrate Visual Signal (VS) and the latent vector that represents Distortion Signal (DS). The distorted input can be reconstructed using both latent space features when merged employing different weight factors. In our method, various types of distortions have been explored. Based on our experiments, the presented method has competitive results with the state-of-the-art with augmented capability. The implementation of our method is available at \url{https://anonymous.4open.science/r/flexible_compression-9E80}