[ milvus-2.4.4 ] error="incomplete query result, missing id xxxxxxx, len(searchIDs) = 298, len(queryIDs) = 291, how to fix it?

[laozhu1900]

[2024/06/13 10:52:55.099 +08:00] [WARN] [proxy/task_search.go:652] ["failed to requery"] [traceID=f500fbe139528473d59e08b8831fc8da] [nq=1] [error="incomplete query result, missing id 202406020002_f83813dceb954132aff00edb2139ffa1_1_A8F79679-8177-194C-8D3D-CA0D82865B48_1, len(searchIDs) = 298, len(queryIDs) = 291, collection=450390723841055357: inconsistent requery result"] [errorVerbose="incomplete query result, missing id 202406020002_f83813dceb954132aff00edb2139ffa1_1_A8F79679-8177-194C-8D3D-CA0D82865B48_1, len(searchIDs) = 298, len(queryIDs) = 291, collection=450390723841055357: inconsistent requery result\n(1) attached stack trace\n  -- stack trace:\n  | github.com/milvus-io/milvus/pkg/util/merr.WrapErrInconsistentRequery\n  | \t/root/code/milvus/pkg/util/merr/utils.go:1038\n  | github.com/milvus-io/milvus/internal/proxy.doRequery\n  | \t/root/code/milvus/internal/proxy/task_search.go:860\n  | github.com/milvus-io/milvus/internal/proxy.(*searchTask).Requery\n  | \t/root/code/milvus/internal/proxy/task_search.go:747\n  | github.com/milvus-io/milvus/internal/proxy.(*searchTask).PostExecute\n  | \t/root/code/milvus/internal/proxy/task_search.go:650\n  | github.com/milvus-io/milvus/internal/proxy.(*taskScheduler).processTask\n  | \t/root/code/milvus/internal/proxy/task_scheduler.go:474\n  | github.com/milvus-io/milvus/internal/proxy.(*taskScheduler).queryLoop.func1\n  | \t/root/code/milvus/internal/proxy/task_scheduler.go:545\n  | github.com/milvus-io/milvus/pkg/util/conc.(*Pool[...]).Submit.func1\n  | \t/root/code/milvus/pkg/util/conc/pool.go:81\n  | github.com/panjf2000/ants/v2.(*goWorker).run.func1\n  | \t/root/goSpace/pkg/mod/github.com/panjf2000/ants/[email protected]/worker.go:67\n  | runtime.goexit\n  | \t/root/packages/go/src/runtime/asm_amd64.s:1695\nWraps: (2) incomplete query result, missing id 202406020002_f83813dceb954132aff00edb2139ffa1_1_A8F79679-8177-194C-8D3D-CA0D82865B48_1, len(searchIDs) = 298, len(queryIDs) = 291, collection=450390723841055357\nWraps: (3) inconsistent requery result\nError types: (1) *withstack.withStack (2) *errutil.withPrefix (3) merr.milvusError"]

milvus version: 2.4.4 , compiled by source code
I upserted 348 pieces of data. when I searched , it reported this ...
How to fix it ?

[yhmo]

Seems it is a known issue: #31642
What kind of SDK/client are you using?

[laozhu1900]

https://github.com/milvus-io/milvus-sdk-java/blob/master/examples/main/java/io/milvus/v1/SimpleExample.java

base on this code,  and use upsert API to add data to colleciton

index Type: IVF_SQ8, metric_type:IP.

primaryKey type is string, UUID

15 columns. including Varchar and int64

[laozhu1900]

I find two new questions.
The collection only has 1218 records .
If my TOPK less than 592, it search correctly. If my TOPK more than 592, it report error .
If I execute flush() , TOPK more than 592, it also search correctly.
why ?

[yhmo]

Is the search() called immediately after data is inserted?

[laozhu1900]

The data was written two days ago

[yhmo]

"TOPK less than 592, it searches correctly. TOPK more than 592, it reports an error ."
It seems to be an unknown bug on the server side. Could you fire an issue here? So we can assign the issue to an engineer to investigate.

[laozhu1900]

邮件已收到，谢谢

[xiaofan-luan]

Java SDK need to implement a retry. @yhmo

[xiaofan-luan]

I think this is an occasional error

[laozhu1900]

But before I execute flush(), if topK more than 592, it reports error every time.
Is it caused by inconsistencies between indexes and data ？

[WheatJH]

i got the same error when i use the hybrid_search;
i have two AnnSearchRequest in the hybrid_search, and the limit are both set 100, and i found when i not use "output_fields" in the hybrid_search, it can get the result; but when i use this param, i have to set the limit in AnnSearchRequest less than the limit in hybrid_search; if i don't do this, i will get the same error

[laozhu1900]

邮件已收到，谢谢

[xiaofan-luan]

this has been fixed and will be released in milvus 2.4.5

[noob9527]

I just encountered the same problem in milvus 2.5.2 .
I tried java sdk version 2.5.2 to 2.5.4, none of them work.

we used 2.4.5 previously, everything works.
today I upgrade to 2.5.2, and recreate my collection from scratch.
now, hybrid search doesn't work.

it throws
incomplete query result, missing id 269794:C:251171:0, len(searchIDs) = 200, len(queryIDs) = 8, collection=455231893840817583: inconsistent requery result

[yhmo]

It is not a SDK-side issue. The latest milvus version still has this problem, hopefully the later versions can fix this issue.

[xiaofan-luan]

this should be fixed by 2.5.3.
please upgrade to latest and let us know if it's working

[laozhu1900]

I am sorry for not replying in time because I have been busy with other things recently...
When I used version 2.4.6, The problem has never been repeated, and I don't know which commit fixed it.
The program was built by source code ( tag-2.4.6 )