The manifold hypothesis states that low-dimensional manifold structure exists in high-dimensional data, which is strongly supported by the success of deep learning in processing such data. However, we argue here that the manifold hypothesis is incomplete, as it does not allow any variation in the intrinsic dimensionality of different sub-regions of the data space. We thus posit the union of manifold hypothesis, which states that high-dimensional data of interest comes from a union of disjoint manifolds; this allows intrinsic dimensionality to vary. We empirically verify this hypothesis on image datasets using a standard estimator of intrinsic dimensionality, and also demonstrate an improvement in classification performance derived from this hypothesis. We hope our work will encourage the community to further explore the benefits of considering the union of manifolds structure in data.