Multi-objective learning problems occur in all aspects of life and have been studied for decades, including in the field of machine learning. Many such problems also exist in distributed settings, where data cannot easily be shared. In recent years, joint machine learning has been made possible in such settings through the development of the Federated Learning (FL) paradigm. However, no general extension of the FL concept to multi-objective learning has been proposed yet, limiting such problems to non-cooperative individual learning. We address this gap by presenting a first general framework for multi-objective federated learning, based on decomposition (MOFL/D). Our framework addresses the a posteriori type of multi-objective problem, where user preferences are not known during the optimisation process, allowing multiple participants to jointly find a set of solutions, each optimised for some distribution of preferences. We present an instantiation of the framework and validate it through experiments on a set of multi-objective benchmarking problems that are extended from well-known single-objective benchmarks.