Skip to content
This repository was archived by the owner on Aug 3, 2026. It is now read-only.

[SE-4304] [SE-4495] feat: add celerybeat configuration - #171

Merged
gabor-boros merged 1 commit into
esme-release/koa.3from
gabor/backport-celery-updates-koa.3
Jul 14, 2021
Merged

[SE-4304] [SE-4495] feat: add celerybeat configuration#171
gabor-boros merged 1 commit into
esme-release/koa.3from
gabor/backport-celery-updates-koa.3

Conversation

@gabor-boros

Copy link
Copy Markdown

The current implementation of celerybeat usage works properly if and
only if one concurrent worker process runs per queue using the nested
beat process started by the --beat flag. In case the the celerybeat
process is running as a separate process not as part of the workers,
that partially solves the problem. Either in that case, if multiple
instances are running in a cluster, without proper process supervising
across instances will result in (at least) dumplicated scheduling,
meaning that the scheduler will cause more work for the workers and
results will be duplicated.

Taking the above into consideration, to solve this issue, this commit
introduces single-beat package that wraps the celerybeat process and
keeps track of the process that acquired the lock in redis. In case one
of the instances get killed or the locking process crashes, the lock

refactor: use boolean flag to enable or disable celery beat
Co-authored-by: Joseph Mulloy jdmulloy@users.noreply.github.com

refactor: use boolean flag to enable or disable celery beat
Co-authored-by: Joseph Mulloy jdmulloy@users.noreply.github.com

Unindent a few entries that needd to be unindented (openedx-unsupported#6450) (openedx-unsupported#6467)
(cherry picked from commit 945fe9d)
Co-authored-by: Chris Pappas christopappas@users.noreply.github.com

Configuration Pull Request

Make sure that the following steps are done before merging

  • A devops team member has commented with 👍
  • are you adding any new default values that need to be overridden when this goes live?
    • Open a ticket (DEVOPS) to make sure that they have been added to secure vars.
    • Add an entry to the CHANGELOG.
  • Are you adding/updating any variables that need to be added/updated to a specific configuration-secure repo?

The current implementation of celerybeat usage works properly if and
only if one concurrent worker process runs per queue using the nested
beat process started by the `--beat` flag. In case the the celerybeat
process is running as a separate process not as part of the workers,
that partially solves the problem. Either in that case, if multiple
instances are running in a cluster, without proper process supervising
across instances will result in (at least) dumplicated scheduling,
meaning that the scheduler will cause more work for the workers and
results will be duplicated.

Taking the above into consideration, to solve this issue, this commit
introduces single-beat package that wraps the celerybeat process and
keeps track of the process that acquired the lock in redis. In case one
of the instances get killed or the locking process crashes, the lock

refactor: use boolean flag to enable or disable celery beat
Co-authored-by: Joseph Mulloy <jdmulloy@users.noreply.github.com>

refactor: use boolean flag to enable or disable celery beat
Co-authored-by: Joseph Mulloy <jdmulloy@users.noreply.github.com>

Unindent a few entries that needd to be unindented (openedx-unsupported#6450) (openedx-unsupported#6467)
(cherry picked from commit 945fe9d)
Co-authored-by: Chris Pappas <christopappas@users.noreply.github.com>

@nizarmah nizarmah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

  • I tested this by testing open-craft/openedx-platform#374.
  • I made sure the cherry-picked commit mentions the commit is was cherry-picked from.
  • I checked for accessibility issues
  • Includes documentation

@gabor-boros
gabor-boros merged commit 129280c into esme-release/koa.3 Jul 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants