Machine learning methods have a groundbreaking impact in many application domains, but their application on real robotic platforms is still limited. Despite the many challenges associated with combining machine learning technology with robotics, robot learning remains one of the most promising directions for enhancing the capabilities of robots. When deploying learning-based approaches on real robots, extra effort is required to address the challenges posed by various factors in the real world. To investigate the key factors influencing real-world deployment and to encourage original solutions from different researchers, we organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We selected the air hockey task as a benchmark, encompassing low-level robotics problems and high-level tactics. Different from other machine learning-centric benchmarks, participants need to tackle practical challenges in robotics, such as the sim-to-real gap, the low-level control issues, the safety problem, real-time requirements, and limited availability to real-world data. The competition's results show that learning-based approaches with prior knowledge integration still outperform the data-driven approaches when building a deployable robotics solution. The ablation study provides us insights into which real-world factors may be overlooked when building a learning-based solution. The real-world air hockey deployment of best-performing agents sets the foundation for future competitions and follow-up research directions.