The size of the collection decreased after the index was created.

[iurii-stepanov]

We loaded 3.3 kk items into the collection, and then created an IVF_FLAT index in it (COSINE, nlist:512). After that, the collection size decreased by 500k items and became 2.7kk. At the same time, I saw such messages in the Index Node logs:

W20241226 11:16:08.675537  2603 ivf.cc:320] [KNOWHERE][MatchNlist][knowhere_build6] nlist(512) is too large, adjust to a proper value
W20241226 11:16:08.675583  2603 ivf.cc:322] [KNOWHERE][MatchNlist][knowhere_build6] Row num 17300 match nlist 443
I20241226 11:16:08.924903    72 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (249.404910 ms)

How can we explain the decrease in the collection after creating the index?
Can the presence of duplicates in the data reduce the size of the collection after the index is created?

[xiaofan-luan]

did you have duplicate primary key?

[iurii-stepanov]

Yes, there were duplicates in the primary key. Thanks for the tip.