In the retail industry, electronic commerce (e-commerce) has grown quickly in the last decade and has further accelerated as a result of movement restrictions during the pandemic. While working with logistics and retail industry business collaborators, we found that the cost of delivery of products from the most opportune node in the supply chain (a quantity called the cost-to-serve or CTS) is a key challenge. We find that a reinforcement learning (RL) formulation is able to exceed the performance of the state of the art rule based policies, while being significantly faster than traditional optimisation approaches such as mixed-integer linear programming. We hypothesize that scaling up the RL based methodology will have a significant impact on the operating margins of retailers in the `new normal'.