Generating novel crystalline materials is a problem of fundamental importance, with a wide range of applications in fields such as electronics, energy storage, and catalysis. Most existing generative models for crystals, learn crystalline symmetries from the training data, similar to other properties. However, given the central roles of symmetry in crystals, it should be front and center in the generative process. To this end, we propose a novel methodology for generating symmetric objects that abandons the common practice of using an invariant prior and an equivariant generation process. Instead, we propose to generate an unconstrained non-symmetric slice of the symmetric object along with its symmetry information. We apply this idea to diffusion-based crystal generation, using an unconstrained generation of the asymmetric unit along with stabilizers for individual atoms within the unit. We demonstrate the effectiveness of our model on MP-20, where various metrics on diversity, validity, and symmetry confirm the competitive performance of our method against the state-of-the-art. This is the first end-to-end deep generative model that generates crystals with desired symmetry without relying on existing templates.