Bayesian optimization (BO) is a powerful tool for optimizing non-convex black-box (also known as derivative-free) functions that are expensive or time-consuming to evaluate and subject to noise in their evaluations. Many important problems can be formulated in this manner, such as optimizing outcomes of high-fidelity computer simulations, automated hyperparameter tuning in machine and deep learning algorithms, A/B testing for website design, policy-based reinforcement learning, and material and drug discovery. In this presentation, three key concepts are introduced, which we argue are critical for enabling and/or improving the practical performance of BO on real-world science and engineering systems. Specifically, one must: (1) leverage prior physics-based knowledge to perform highly efficient (targeted) exploration of the solution space; (2) explicitly incorporate safety constraints during interaction with physical systems to avoid unsafe, unethical, and/or undesirable outcomes; and (3) account for external sources of uncertainty during the search process to ensure the best-identified solution is robust/flexible in practice. We discuss a unified framework for adapting BO to handle these considerations and illustrate how this framework can be deployed in practice on a series of examples ranging from the design of safe cold plasma jet devices to the discovery of high-performance sustainable energy storage materials. We also offer perspectives on key challenges and future opportunities in the realm of applied BO.