Remain Well Clear, keeping the aircraft away from hazards by the appropriateseparation distance, is an essential technology for the safe operation of uncrewedaerial vehicles in congested airspace. This work focuses on automating the horizontal separation of two aircraft and presents the obstacle avoidance problem as a2D surrogate optimization task. By our design, the surrogate task is made moreconservative to guarantee the execution of the solution in the primary domain.Using Reinforcement Learning (RL), we optimize the avoidance policy and modelthe dynamics, interactions, and decision-making. By recursively sampling theresulting policy and the surrogate transitions, the system translates the avoidancepolicy into a complete avoidance trajectory. Then, the solver publishes the trajectoryas a set of waypoints for the airplane to follow using the Robot Operating System(ROS) interface.The proposed system generates a quick and achievable avoidance trajectory thatsatisfies the safety requirements. Evaluation of our system is completed in a high-fidelity simulation and full-scale airplane demonstration. Moreover, the paperconcludes an enormous integration effort that has enabled a real-life demonstrationof the RL-based system.