De novo protein design has been greatly accelerated by the advent of generative models of protein structure. While more coarse-grain tasks such as backbone generation are increasingly possible, existing models still struggle on problems that require precise placements at the atomic scale, motivating the development of all-atom generative models. However, all-atom generative modeling remains challenging due to the complex interplay between discrete and continuous aspects of protein structure. In this work we propose a framework to capture this interplay by combining backbone generation methods using flow-matching on backbone frames with flow-matching in learned latent spaces. We present a prototype of our method ProteinZen that implements these ideas, and demonstrate promising initial results on the all-atom generation task, with 46% of samples being sequence-structure consistent at the atomic level while retaining competitive sample diversity and novelty.