The use of automatic speech recognition (ASR) systems for spoken languages has become widespread recently. Contrarily, the vast majority of African languages have limited linguistic resources to sustain the robustness of these systems. We present a study on an end-to-end speech recognition system for Nigerian-Pidgin-English. Using our unique dataset, we fine-tuned different variants of the Wac2Vec2.0 architecture. We contrasted the results of these techniques with those of preceding studies. Empirically, we achieved a low word error rate of 33\% on the test set outperforming the baseline method and also surpassed other variants of the Wac2Vec2.0 architecture in terms of qualitative assessments.