WiFi sensing is an emerging technology that uses wireless signals for sensing applications like human activity recognitions, but challenges like limited labeled data and complexity of channel state information (CSI) data, hinder model performance and generalization. We propose Context-Aware Predictive Coding (CAPC), a novel self-supervised learning (SSL) framework for CSI-based WiFi sensing. CAPC integrates temporal contrastive prediction with an augmentation-based contextual approach, which captures temporal dependencies while eliminating CSI estimation and transceiver errors through a proposed dual view augmentation method. The combined strategy ensures that CAPC learns robust and informative representations while maintaining temporal coherence and contextual consistency. CAPC outperforms supervised and SSL baselines, especially with limited labeled data from unseen environments and in transfer learning to new datasets and tasks, making it a strong alternative for WiFi sensing applications.