Puck proposes a multi-level index structure towards the ever growing datasets in practice. Two key methods, dynamic trimming and unifying PQ-table, are designed to accelerate training and search of multi-level quantization-based index. While maintaining a recall ratio >95% towards the SIFT10M dataset, a four-level quantization-based index achieves significantly better search throughput than Faiss-ivf and HNSW. The repository in GitHub is https://github.com/baidu/puck .

Authors： Jie Yin, Baidu Ben Huang, Baidu