Irish potato is among the important food and cash crops to most smallholder farmers in Tanzania. Despite its importance in household economy and food security, yields are generally low due to the effects of diseases, specifically Early and Late blight. The current management of these two diseases includes the removal of the affected leaves and plants to reduce their spread, signifying that early detection is the key to successful management. This study therefore developed a Machine Learning model to detect early these two diseases based on leaf imagery data and enable the farmer to make appropriate decision for managing the spread of the diseases. Resnet152 and Inceptionv3 Convolution Neural Network architectures were used to train the model in a dataset of 50,310 imagery samples. The results showed that Resnet152 achieved an accuracy of 83.4% while Inceptionv3 achieved an accuracy of 80.1%. These results demonstrate the suitability of our model to early detect Early and Late blight diseases.