In online applications with streaming data, awareness of how far the empirical training or test data has shifted away from its original data distribution can be crucial to the performance of the model. However, historical samples in the data stream may not be kept either due to space requirements or for regulatory reasons. To cope with such situations, we propose Continual Density Ratio Estimation (CDRE), for estimating density ratios between the initial and latest distributions (p/qt) of a data stream without the need of storing past samples, where qt shifted away from p after a time period t. In particular, CDRE is more accurate than standard DRE when the two distributions are less similar, despite not requiring samples from the original distribution. CDRE can be applied in scenarios of online or continual learning, such as importance weighted covariate shift, measuring dataset changes for better decision making.