Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In 2.x, seems series ids may be possible to be incomplete in series id set cache? #25057

Open
Rachelint opened this issue Jun 12, 2024 · 0 comments

Comments

@Rachelint
Copy link

Rachelint commented Jun 12, 2024

Steps to reproduce:
I am reading 2.x source codes recently, and found series ids seems can be incomplete in some cases? I am not so sure about it...

The tagValueCache's normal filling path seems like:

  • Search series ids by tag key + tag value from each partition(include current inactive log file, active log file, index file) in Index::TagValueSeriesIDIterator.
  • Build the multi level index measurement -> tag key -> tag value -> series ids in Index::TagValueSeriesIDIterator.
  • Append the new log(include new series and its index entries) into active log file in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.
  • Insert the new series id if the measurement -> tag key -> tag value -> series ids path above exist in cache in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.

However, due to multithreading, the order of steps can be:

  • Search series ids by tag key + tag value from each partition(include current inactive log file, active log file, index file) in Index::TagValueSeriesIDIterator.
  • Append the new log(include new series and its index entries) into active log file in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.
  • Try to Insert the new series id, but found the measurement -> tag key -> tag value -> series ids path non-exist in cache in Index::CreateSeriesIfNotExists or index::CreateSeriesListIfNotExists.
  • Build the multi level index measurement -> tag key -> tag value -> series ids in Index::TagValueSeriesIDIterator.

And in the later query, because the measurement -> tag key -> tag value -> series ids exists, it won't try to search from files again, so the series ids have no changes to become complete...

Expected behaviour:
Complete series ids in cache.

Actual behaviour:
Maybe incomplete series ids in cache.

Environment info:

Config:

Logs:

@Rachelint Rachelint changed the title In 2.x, It seems series ids maybe possible to be incomplete in series id set cache? In 2.x, It seems series ids may be possible to be incomplete in series id set cache? Jun 12, 2024
@Rachelint Rachelint changed the title In 2.x, It seems series ids may be possible to be incomplete in series id set cache? In 2.x, seems series ids may be possible to be incomplete in series id set cache? Jun 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant