Directed evolution is a powerful but resource-intensive method for optimizing protein functionalities by screening a vast range of mutations. Recent advances in machine learning aim to streamline this process by using surrogate sequence-function models. We propose Latent-based Directed Evolution (LDE), an evolutionary algorithm that efficiently explores high-fitness mutants in the latent space. Built on a regularized variational autoencoder (VAE) and the state-of-the-art Protein Language Model ESM-2, LDE creates a meaningful latent representation of sequences and combines gradient-based methods with directed evolution for effective fitness landscape traversal. Our experimental results on eight protein design tasks show that LDE outperforms existing baseline algorithms. Our source code is publicly available at https://github.com/HySonLab/LatentDE