Accurate prediction of protein-ligand binding pockets is a critical task in protein functional analysis and small molecule pharmaceutical design. However, the flexible and dynamic nature of proteins conceal an unknown number of potentially invaluable "cryptic" pockets. Current approaches for cryptic pocket discovery rely on molecular dynamics (MD), leading to poor scalability and bias. Even recent ML-based cryptic pocket discovery approaches require large, post-processed MD datasets to train their models. In contrast, this work presents ``Efficient Sequence-based cryptic Pocket prediction'' (ESP) leveraging advanced Protein Language Models (PLMs), and demonstrates significant improvement in predictive efficacy compared to ML-based cryptic pocket prediction SOTA (ROCAUC 0.93 vs 0.87). ESP achieves detection of cryptic pockets via training on readily available, non cryptic-pocket-specific data from the PDBBind dataset, rather than costly simulation and post-processing. Further, while SOTA's predictions often include positive signal broadly distributed over a target structure, ESP produces more spatially-focused predictions which increase downstream utility.