The task of math word problems has recently shown encouraging progress, e.g. in Recall and Learn (REAL), that solving problem by retrieving most similar questions based on a pre-trained memory module. In this article, we verify the effectiveness of different neural memory modules that can be trained end-to-end. Specifically, we first propose a Top-N pre-ranking process to retrieve candidate questions based on a Word2Vec model, and then we utilize a trainable memory module to re-rank the candidates to obtain the most similar Top-K questions. With this simple modification, we establish a stronger framework REAL2 that achieves state-of-the-art results. Code will be made public and we hope it will make the research of analogical learning in MWP task more accessible.