We investigate the role of cognitive maps and hippocampal-entorhinal architecture in a mental navigation (MNAV) task by conducting experiment in humans, monkeys and neural network models. Humans can generalize their mental navigation performance to untrained start-target landmark pairs in a given landmark sequence and also rapidly adapt to new sequences. The model uses a continuous-time recurrent neural network (CTRNN) for action decisions and a hippocampal-entorhinal model network, MESH (Memory network with Scaffold and Heteroassociation), for encoding and learning maps. The model is first trained on a navigation-to-sample (NTS) task and tested on MNAV task where no sensory feedback is available, across five different environments (i.e. landmark sequences). The CTRNN with MESH solves MNAV task by reconstructing the next image via path integration and vastly outperforms the model with CTRNN alone. In both NTS and MNAV tasks, MESH-CTRNN model shows better generalization to untrained pairs within each environment and faster adaptation to new environments. Like humans, monkeys also exhibit generalization to untrained landmark pairs in MNAV task. We compared the neural dynamics in monkeys' entorhinal cortex to the dynamics of CTRNN and found behaviorally relevant periodic signals in both. The study demonstrates the importance of hippocampal cognitive maps in enabling data-efficient and generalizable learning in the brain.