There is an increasing drive to integrate machine learning (ML) tools into the drug development pipeline, to improve success rates and efficiency in the clinical development pathway. The ML regulatory framework being developed is closely aligned with ML best practices. However, there remain significant and tangible practical gaps in translating best practice standards into a real-world clinical trial context.To illustrate the practical challenges to regulating ML in this context, we present a theoretical oncology trial in which a ML tool is applied to support toxicity monitoring in patients. We explore the barriers in the highly regulated clinical trial environment to implementing data representativeness, model interpretability, and model usability.