Particle accelerators require continuous adjustment to maintain beam quality. At the Advanced Photon Source (APS) synchrotron facility this is accomplished using a mix of operator-controlled and automated tools. We have recently implemented Bayesian optimization (BO) as one of automated options, significantly improving sampling efficiency. However, poor BO performance was observed in certain scenarios due to time-dependent device drifts. In this work, we discuss extending BO to an adaptive version (ABO) that can compensate for distribution drifts through explicit time-awareness, enabling long-term online operational use. Our contributions include advanced kernels with physics-informed time dimension structure, age-biased data history subsampling, and auxiliary time-aware safety constraint models. Benchmarks show better ABO performance in several simulated and experimental tests. Our results are an encouraging step for the wider adoption of ML-based optimizers at APS.