Inspired by ConvNets leveraging hierarchical representations, we introduce Tensor-based ConvNets (TConvNets) employing hierarchical neurons. TConvNets, a more generalized form of ConvNets, necessitate a generalized version of components and operations. Unlike ConvNets with scalar neurons, TConvNets use tensor-based neurons, relying on tensor production and combination as core operations instead of linear combinations. Key components, including tensor-based batch normalization and initialization, are developed for TConvNets. Additionally, methods for structuring/unstructuring input/output allow the utilization of ConvNets components like loss functions in TConvNets. Although TConvNets may offer many new attributes, this paper focuses solely on parameter-wise efficiency. Through constructing a TConvNet with high-rank neuron tensors, we conducted performance comparisons on CIFAR10, CIFAR100, and Tiny ImageNet datasets, revealing TConvNets' superior efficiency in parameter utilization.