can not overwrite index file '/opt/homebrew/var/manticore/t/wf'

[sanikolaev]

Bug Description:

When I alter a table with the same wordforms file as previously, it fails:

MRE

➜  ~ echo "a > b" > /tmp/wf; mysql -P9306 -h0 -v -e "drop table if exists t; create table t(f text) wordforms='/tmp/wf'; alter table t wordforms='/tmp/wf'"
--------------
drop table if exists t
--------------

--------------
create table t(f text) wordforms='/tmp/wf'
--------------

--------------
alter table t wordforms='/tmp/wf'
--------------

ERROR 1064 (42000) at line 1: can not overwrite index file '/opt/homebrew/var/manticore/t/wf'

Expected: it doesn't fail.

Manticore Search Version:

Manticore 6.3.0 1811a9efb@24052209 (columnar 2.3.0 88a01c3@24052206) (secondary 2.3.0 88a01c3@24052206) (knn 2.3.0 88a01c3@24052206)

Operating System Version:

macos

Have you tried the latest development version?

No

Internal Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed
• Changelog updated

[tomatolog]

it worth to add some counter like chunk id or any other uniq seq as external file suffix on coping the file. That should fix the clash and allow alter table t wordforms ... to finish well even it add the same external file multiple times

[tomatolog]

just pushed the fix into the branch https://github.com/manticoresoftware/manticoresearch/tree/ext_files_naming however it could be better to create clt test for the external files case to make sure create table and alter table works as usual. The cases are:

the create table cases:

• create table with different externals such as
  • stopwords (multiple stopwords file in the same string option ) like create table ... stopwords = 'en, it, /tmp/some_stops'
  • exceptions like create table ... exceptions= '/tmp/some_exc'
  • wordforms (multiple wordforms options ) like create table ... wordforms= '/tmp/wf1' wordforms= '/tmp/wf2' wordforms= '/tmp/some_mask*'

the show index table settings should show all these external files and daemon restart there should be no warning about missed external files and show index table settings should show all these external files

the alter table cases:

• alter table should replace the stopwords \ exceptions \ wordforms with the new files and also remove the option for the empty option string like alter table tbl stopwords='' should remove stopwords from the table
• that should persist the daemon restart
• the alter cases should be done for RT index with only RAM part and also for RT index with disk chunks

If tests show no additional regressions we could merge that branch into the master.

[sanikolaev]

@PavelShilin89 pls do manual testing first and prepare CLT tests.

[PavelShilin89]

Done in #2373

[sanikolaev]

@PavelShilin89 pls remake the PR as it includes changes related with Logstash and other integrations that are unrelated with this issue.

[PavelShilin89]

Done in #2359

[sanikolaev]

@tomatolog pls review Pavel's CLT tests in your branch to make sure nothing is missed from your tests spec

[tomatolog]

I merged PR as new cases covered by CLT tests and failed CLT tests are also fixed.