Temporal hidden Markov models (HMMs) and tensor trains are two powerful mathematical tools for learning and representing high-dimensional large-scale data. However, both HMMs and tensor trains have temporal topology but are not ergodic. Ergodicity occurs in a broad range of systems in physics and in geometry, and many instances of real-world data originate from an ergodic system that exhibits temporal behavior. To overcome the fundamental limitations of temporal HMMs and tensor trains, i.e., ergodicity deficiency, we propose a new tensor topology inspired by tensor rings and circular HMMs. Our new tensor ring models-- namely Circular Matrix Product State (CMPS), Circular Born Machine (CBM), and Circular Locally Purified State (CLPS)-- are both temporal and ergodic. Then, we show the relationship between our tensor ring models and circular hidden Markov models/hidden quantum Markov models. Our findings through numerical experiments indicate that our tensor ring models have significant advantages over tensor trains in varied situations including cases where ergodicity is not required.