In real life, the decoration of 3D indoor scenes through designing furniture layoutprovides a rich experience for people. In this paper, we explore the furniturelayout task as a Markov decision process (MDP) in virtual reality, which is solvedby hierarchical reinforcement learning (HRL). The goal is to produce a propertwo-furniture layout in the virtual reality of the indoor scenes. In particular, wefirst design a simulation environment and introduce the HRL formulation for atwo-furniture layout. We then apply a hierarchical actor-critic algorithm withcurriculum learning to solve the MDP. We conduct our experiments on a large-scalereal-world interior layout dataset that contains industrial designs from professionaldesigners. Our numerical results demonstrate that the proposed model yieldshigher-quality layouts as compared with the state-of-art models.