Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: fix INIT state stuck #227

Merged
merged 1 commit into from
Apr 15, 2024
Merged

Conversation

DerekBum
Copy link
Contributor

@DerekBum DerekBum commented Apr 4, 2024

Sometimes, instance could enter the queue initialization while still not running (for example, left in the orphan mode). This resulted in "lazy start". But Tarantool does not call box.cfg {} after leaving orphan mode, so queue could stuck in the INIT state.

Now if the instance is read-only, separate fiber is watching for updates of its mode.

Note that this fix works only for Tarantool versions >= 2.10.0. This is because of used watchers.

Closes #226

Copy link
Collaborator

@oleg-jukovec oleg-jukovec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to block queue_init() until orphan mode switched? The wait and the queue initialization could be done in a background as it is done by box.cfg wrapper now.

queue/init.lua Outdated Show resolved Hide resolved
@DerekBum
Copy link
Contributor Author

DerekBum commented Apr 8, 2024

The wait and the queue initialization could be done in a background as it is done by box.cfg wrapper now.

Tarantool does not call box.cfg {} upon exiting the orphan mode. Or are you suggesting to run a fiber in a background, that will check status of the instance. And if the orphan mode changes, it updates the status?

@oleg-jukovec
Copy link
Collaborator

The wait and the queue initialization could be done in a background as it is done by box.cfg wrapper now.

Tarantool does not call box.cfg {} upon exiting the orphan mode. Or are you suggesting to run a fiber in a background, that will check status of the instance. And if the orphan mode changes, it updates the status?

Yes, it is. When the status check will success - initialize the instance in a background.

@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch 4 times, most recently from 2c94e6a to c42768c Compare April 9, 2024 10:12
@DerekBum
Copy link
Contributor Author

DerekBum commented Apr 9, 2024

Updated the code. Now watcher is run in a different fiber, without blocking init function. Note that now this fix only works for Tarantool versions >= 2.10.0. On newer versions this bug will still be present.

CHANGELOG.md Outdated Show resolved Hide resolved
queue/init.lua Show resolved Hide resolved
queue/init.lua Outdated Show resolved Hide resolved
queue/init.lua Outdated Show resolved Hide resolved
@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch 2 times, most recently from 5ebc059 to 854a373 Compare April 9, 2024 11:58
@oleg-jukovec oleg-jukovec requested a review from 0x501D April 9, 2024 12:18
queue/init.lua Outdated Show resolved Hide resolved
@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch from 854a373 to 2cb75ad Compare April 9, 2024 12:42
queue/init.lua Outdated Show resolved Hide resolved
Copy link
Member

@0x501D 0x501D left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch 4 times, most recently from 0524b35 to b07da0c Compare April 10, 2024 17:21
@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch 2 times, most recently from 155dc3f to 8a440ed Compare April 11, 2024 14:02
t/230-orphan-not-stalling-init.t Outdated Show resolved Hide resolved
t/230-orphan-not-stalling-init.t Outdated Show resolved Hide resolved
@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch from 8a440ed to 9eb38b3 Compare April 11, 2024 17:00
queue/init.lua Outdated Show resolved Hide resolved
queue/init.lua Outdated Show resolved Hide resolved
queue/init.lua Outdated Show resolved Hide resolved
@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch from 9eb38b3 to dad5971 Compare April 15, 2024 11:15
queue/init.lua Outdated Show resolved Hide resolved
@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch 2 times, most recently from 59a4032 to 4bdc2dd Compare April 15, 2024 14:19
Sometimes, instance could enter the queue initialization
while still not running (for example, left in the orphan mode).
This resulted in "lazy start". But Tarantool does not call
`box.cfg {}` after leaving orphan mode, so queue could stuck in the
`INIT` state.

Now if the instance is read-only,  separate fiber is watching for
updates of its mode.

Note that this fix works only for Tarantool versions >= 2.10.0.
This is because of used watchers.

Closes #226
@DerekBum DerekBum force-pushed the DerekBum/gh-226-fix-init-state-stuck branch from 4bdc2dd to 530c026 Compare April 15, 2024 17:00
@oleg-jukovec oleg-jukovec merged commit aa7c092 into master Apr 15, 2024
23 checks passed
@oleg-jukovec oleg-jukovec deleted the DerekBum/gh-226-fix-init-state-stuck branch April 15, 2024 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

queue stuck in INIT state on tarantool 3
3 participants