This paper proposes an optimization-based method to learn the singular value decomposition (SVD) of a compact operator with ordered singular functions. The proposed objective function is based on Schmidt's low-rank approximation theorem (1907) that characterizes a truncated SVD as a solution minimizing the mean squared error, accompanied with a technique called \emph{nesting} to learn the ordered structure. When the optimization space is parameterized by neural networks, we refer to the proposed method as \emph{NeuralSVD}. The implementation does not require sophisticated optimization tricks unlike existing approaches.