In data-driven applications, it is increasingly desirable to collect data from different sources for enhancing performance.In this paper, we are interested in the problem of probabilistic forecasting with multi-source time series.We propose a neural mixture structure-based probability model for learning different predictive relations and their adaptive combinations from multi-source time series.We present the prediction and uncertainty quantification methods, which are applicable to different distributions of target variables.Additionally, given the imbalanced and unstable behaviors observed during the direct training of the proposed mixture model, we develop a phased learning method and provide a theoretical analysis.In the experimental evaluation, the mixture model trained by the phased learning exhibits competitive performance on both point and probabilistic prediction metrics.Meanwhile, the proposed uncertainty conditioned error suggests the potential of the mixture model's uncertainty score as a reliability indicator of predictions.