Although Bayesian optimization (BO) is commonly used to optimize many scientific and industrial experiments, it needs specialized acquisition functions (AF) to navigate exploration spaces efficiently for various application domains. Moreover, traditional BO struggles to incorporate prior data from simulations or previous experiments, resulting in lower sample efficiency. This paper explores freely available LLMs, fine-tuned to generate Python code, to refine acquisition functions while learning from simulations and experiments. Our results show that this method generates novel acquisition functions that beat traditional BO methods like EI and UCB. Inertial confinement fusion (ICF) is costly and requires complex optimization of the laser pulse shape across multiple shots to maximize yield. Our adaptation of this LLM-based technique shows that it outperforms classic AFs such as EI and UCB in ICF optimization, leading to more efficient and effective optimization and accelerating scientific innovation.