With the explosive growth of available training data, single-image 3D human modeling is ahead of a transition to a data-centric paradigm.A key to successfully exploiting data scale is to design flexible models that can be supervised from various heterogeneous data sources produced by different researchers or vendors.To this end, we propose a simple yet powerful paradigm for seamlessly unifying different human pose and shape-related tasks and datasets.Our main design goal is the ability - both at training and test time - to query any arbitrary point of the human volume, and obtain its estimated location in 3D.We achieve this by learning a continuous neural field of body point localization functions, each of which is a differently parameterized 3D heatmap-based convolutional point localizer (detector).Through this, we can naturally exploit differently annotated data sources including mesh, 2D/3D skeleton and dense pose, without having to convert between them, and thus train large-scale 3D human mesh and skeleton estimation models that outperform the state-of-the-art on several public benchmarks including 3DPW, EMDB and SSP-3D by a considerable margin.