Protein macromolecules are known to play key roles in cellular processes. Solving inverse design problems can allow us to control targeted cellular processes by designing proteins optimized for downstream tasks. However, current fixed-backbone protein design methods are limited to generating one type of secondary structure for a set of design candidates, that are learned from distributions of a single modality (either sequence or structure). To this end, we propose a diffusion-based generative modelling method that co-designs sequence and structure properties for an arbitrary distribution of proteins structures by optimizing over a function of a downstream protein task. We demonstrate preliminary results of an equivariant joint diffusion process for 2 modalities, with the goal of scaling to more modalities.