Many time series classification tasks where labels vary over time are affected by label noise that also varies over time. Such noise can cause label quality to improve, worsen, or periodically change over time. We first propose and formalize temporal label noise, an unstudied problem for sequential classification of time series. In this setting, multiple labels are recorded in sequence while being corrupted by a time-dependent noise function. We demonstrate the importance of modelling the temporal nature of the label noise function and how existing methods consistently underperform. We then demonstrate the surprising noise tolerance of time series foundation models and how this collapses under temporal label noise.