Multi-fidelity Bayesian Optimization (MFBO) is a promising framework to speedup materials and molecular discovery as sources of information of different accuracies are at hand at increasing cost. Despite its potential use in chemical tasks,there is a lack of systematic evaluation of the many parameters playing a rolein MFBO. In this work, we provide guidelines and recommendations to decidewhen to use MFBO in experimental settings. We investigate MFBO methodsapplied to molecules and materials problems. First, we test two different familiesof acquisition functions in two synthetic problems and study the effect of theinformativeness and cost of the approximate function. We use our implementationand guidelines to benchmark three real discovery problems and compare themagainst their single-fidelity counterparts. Our results may help guide future effortsto implement MFBO as a routine tool in the chemical sciences.