Mood disorders are increasingly recognized among the leading causes of disease burden worldwide. Depressive and manic episodes in mood disorders commonly involve altered mood, sleep, and motor activity. These translate to changes in sensory data that wearable devices can continuously and affordably monitor, thereby positioning themselves as promising candidate to model mood disorders. Previous similar endeavors cast this problem in terms of binary classification (cases vs controls) or regress the total score of some commonly used psychometric scale. Nevertheless, these approaches fail to capture the variability within symptom domains described at the item level in psychometric scales. In this work, we attempt to infer mood disorder symptoms (e.g., depressed mood, insomnia, irritability) from time-series data collected with the medical grade Empatica E4 wristbands, as part of an exploratory, observational, and longitudinal study. We propose a multi-label framework to predict individual items from the two most widely used scales for assessing depression and mania. We experiment with two different approaches to preprocess the high-dimensional and noisy sensory data and attain results within a clinically acceptable level of error.