In this study, we introduce a novel approach for enhancing the use of Optimal Transport (OT) in analysing gene expression trajectories within single-cell RNA-seq data. In contrast to existing methods that often depend on linear embeddings or Gaussian autoencoder latent spaces, our approach, performing OT-based trajectory inference on statistical manifolds, accounts for critical data characteristics such as sparsity, overdispersion, and geometry. We achieve this by implementing a "flattening" regularisation derived from the pullback metric of a negative binomial statistical manifold, ensuring alignment between the latent space of a discrete Variational Autoencoder (VAE) and Euclidean space, thereby improving compatibility with linear OT. Our real data results demonstrate that these constraints benefit the reconstruction of latent trajectories and the learning of velocity fields. We believe that this versatile approach holds promise for advancing single-cell representation learning and temporal modelling in the future.