Introduce Mutex Map in IndexInfo to Ensure Thread Safety During Index Construction

[Beihao-Zhou]

Search before asking

• I had searched in the issues and found no similar issues.

Motivation

In the context of index, the absence of a locking mechanism could lead to issues when multiple threads attempt to modify shared resources simultaneously.

[1] Original issue raised from HNSW #2481
[2] Marked TODO

kvrocks/src/server/redis_connection.cc

Lines 522 to 528 in 3408318

	// TODO: transaction support for index updating
	for (const auto &record : index_records) {
	auto s = GlobalIndexer::Update(record);
	if (!s.IsOK() && !s.Is<Status::TypeMismatched>()) {
	LOG(WARNING) << "index updating failed for key: " << record.key;
	}
	}

We might want to introduce locks to solve the problem.

Solution

Solution 1: Mutex (Straight-forward)

We can introduce a mutex map within each IndexInfo structure. This mutex map would contain a mutex for each field that requires protection against concurrent access.

Con

• Blocking if multiple connections update the same index

Solution 2: Queue

We can store the Update task for each IndexInfo field in a queue, and have a background thread to schedule tasks in order asynchronously. I'm not quite sure about how many number of threads would be appropriate, but can discuss further if everyone leans towards this solution.

cc @PragmaTwice

Are you willing to submit a PR?

• I'm willing to submit a PR!

[PragmaTwice]

We can first try to implement the first method because it is simpler and does ensure correctness of indexing.

We can then benchmark to determine its impact on performance and consider how to further optimize it.