Causal Optimal Transport (COT) results from imposing a temporal causality constraint on classic optimal transport problems.Relying on recent work of COT-GAN optimized for sequential learning, the contribution of the present paper is twofold. First, we develop a conditional version of COT-GAN suitable for sequence prediction. This means that the dataset is now used in order to learn how a sequence will evolve given the observation of its past evolution. Second, we improve on the convergence results by working with modifications of the empirical measures via kernel smoothing. The resulting kernel conditional COT-GAN (KCCOT-GAN) algorithm is illustrated with an application for video prediction.