We develop ProtComposer to generate protein structures conditioned on spatial protein layouts that are specified via a set of 3D ellipsoids capturing substructure shapes and semantics. At inference time, we condition on ellipsoids that are hand-constructed, extracted from existing proteins, or generated heuristically, with each option unlocking new capabilities. Hand-specifying ellipsoids enables users to \emph{control} the location, size, orientation, secondary structure type, and approximate shape of protein substructures. Conditioning on ellipsoids of existing proteins enables redesigning their substructure's connectivity or \emph{editing} substructure properties. By conditioning on novel and diverse ellipsoid layouts produced by a simple statistical model, we improve protein generation with expanded Pareto frontiers between designability, novelty, and diversity. Further, this enables sampling designable proteins with a helix-fraction that matches natural proteins, unlike existing generative models that commonly oversample conceptually simple helix bundles.