manticore 6.3.2 confuses in which field given string is

[pakud]

Bug Description:

to reproduce run:

cat l2.sql | mysql -h127.0.0.1 -P9306
echo "SELECT id, content, contentuninteresting FROM test WHERE MATCH('@(content) AnalysisSettings') LIMIT 1\G"|mysql -h127.0.0.1 -P9306

inspect the output - it does not contain AnalysisSettings in the content column, on the other hand column contentuninteresting contains that substring, but MATCH should ignore it.

this issue seems to be nondeterministic - affects different rows on different times, but generally i can reproduce it with every full load of l2.sql

l2.zip

thanks in advance for investigating it!

Manticore Search Version:

6.3.2-24062606-c296dc7c8

Operating System Version:

Debian GNU/Linux 12

Have you tried the latest development version?

No

Internal Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed
• Changelog updated

[sanikolaev]

Thanks @pakud !

I can reproduce it with:

CREATE TABLE test(id bigint, content text,contentuninteresting text, subject text) hitless_words='all';

but w/o the hitless_words it doesn't fail, so it has smth to do with that.

highlight() doesn't highlight it which is ok, just a confirmation we are not missing smth and the keyword is indeed found in the wrong field:

# echo "SELECT highlight() FROM test WHERE MATCH('@content AnalysisSettings') LIMIT 1\G"|mysql -h127.0.0.1 -P9306

*************************** 1. row ***************************
highlight(): e79c111 a21d62c 7a47627ae6be 77 d2662e9 78 2 bbb2e9f 8f236f581408 a 7 d97562 f3c3 a4 a15 2 e2a1dea01bb4cd a 7 818b6f4 2f5 7 dff98a3 2f5 ea35d db58a c94be3b8372e94dfbab730 923578 ef227e456db9ae651a98df 18bd2ef344e4c31794 7 35c 07421fb3393858 8693 47d23be  ...  |   ...  | 246b56d0 ...

[tomatolog]

ok I reproduced the case and investigating it

[tomatolog]

should be related to #569 RT index writes disk chunk with hitless wrong, ie while data in memory searches works well - after it got saved as disk chunk it becomes invalid and search got broken that appears as @field term query returns result that should be not be there

[tomatolog]

I've just fixed this case at the c811e4e now RT index saves disk chunk with right format and result set should be the same for query into RT RAM or RT that saves data into disk chunk.

However you need to truncate and reindex your data from skratch after you updates daemon from the dev repository to get the issue fixed.

[pakud]

@tomatolog , @sanikolaev - thanks a lot for the fix - it works fine!

i've tested it with nightly dev build 6.3.3-24071212-0f9c12b53 on production data having more rows and columns.