Extracting physical dynamical system parameters from videos, which is important for scientific and technological applications. Current methods rely on supervised deep networks trained on large labeled datasets which are difficult to acquire. Existing unsupervised techniques, which rely on frame prediction, have limitations such as long training times, instability, and applicability to specific motion problems. The proposed method addresses these issues by estimating physical parameters of any known, continuous governing equation from single videos using a KL-divergence-based loss function in the latent space. This approach is robust, works for various systems beyond motion, and eliminates the need for frame prediction, reducing model size and computation.