In many machine learning applications, data are collected by a large number of devices, calling for a distributed architecture for learning models. Federated learning (FL) aims to address this challenge by providing a decentralized mechanism for leveraging the individual data and computational power of users. Classical FL relies on a single shared model for users but tends to perform poorly in the presence of data and task heterogeneity across users.

This talk presents various approaches for developing multiple ``personalized” models for heterogeneous users. We first consider a meta-learning approach, where the goal is to generate an initial shared model that users adapt to their tasks using small number of additional local computations. Second, we consider a cluster-based approach which is more appropriate when there is substantial heterogeneity in user data distributions. We propose an algorithm that simultaneously learns cluster identities, while fully operating in a decentralized manner.