Deep learning and machine learning have recently attracted remarkable attention in the inverse design of nanostructures. However, limited works have used these techniques to reduce the design complexity of structures. In this work, we present an evolutionary-based method using manifold learning for inverse design of nanostructures with minimal design complexity. This method encodes the high dimensional spectral responses obtained by electromagnetic simulation software for a class of nanostructure with different design complexities using an autoencoder (AE). We model the governing distributions of the data in the latent space using Gaussian mixture models (GMM) which then provides the level of feasibility of a desired response for each structure and use a neural network (NN) to find the optimum solution. This method also provides valuable information about the underlying physics of light-matter interactions by representing the sub-manifolds of feasible regions for each design complexity level (i.e., number of design parameters) in the latent space. To show the applicability of the method, we employ this technique for inverse design of a class of nanostructures consisting of dielectric metasurfaces with different complexity degrees.