Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add -c/--cluster flag to xdmod-slurm-helper #1401

Merged
merged 3 commits into from
Aug 28, 2020

Conversation

treydock
Copy link
Contributor

@treydock treydock commented Jul 28, 2020

Description

Adds -c/--cluster flag to xdmod-slurm-helper.

Motivation and Context

At OSC for a short period we have a new expansion coming online that is SLURM. The original cluster is Torque. To allow both in XDMOD we created a second resource called pitzer-exp but the actual SLURM cluster name is pitzer. Later this year we will merge the Torque Pitzer into the SLURM expansion Pitzer and both resource name and SLURM cluster name will be pitzer. This change just allows the flexibility to have the cluster name in SLURM differ from the resource name in XDMOD.

Tests performed

Been running this change for a about a week on OSC's XDMOD test host that shreds and ingests production data.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project as found in the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@treydock
Copy link
Contributor Author

This is what my cron execution looks like to make it a bit clearer how this is used:

15 1 * * * xdmod /usr/bin/xdmod-slurm-helper --quiet -r pitzer-exp -c pitzer

@jtpalmer
Copy link
Contributor

@treydock Our current plan is to not add any new functional changes before the 9.0 release. Is that going to cause problems for you? You can produce the same result as your change by running sacct and xdmod-shredder instead of using the helper script.

@jtpalmer jtpalmer added Category:ETL Extract Transform Load enhancement Enhancement of the functionality of an existing feature new feature New functionality labels Jul 28, 2020
@treydock
Copy link
Contributor Author

I can maintain a patch until 9.5, that's not a problem. I was going to do sacct + xdmod-shredder but didn't want to deal with the complicated nature of having correct timestamps and such. It was easier to patch the helper script than to build up necessary sacct command in cron.

@jtpalmer jtpalmer added this to the 9.5.0 milestone Aug 5, 2020
@plessbd plessbd changed the base branch from xdmod9.0 to xdmod9.5 August 19, 2020 12:43
bin/xdmod-slurm-helper Outdated Show resolved Hide resolved
@jtpalmer jtpalmer merged commit a917376 into ubccr:xdmod9.5 Aug 28, 2020
@jtpalmer jtpalmer removed the enhancement Enhancement of the functionality of an existing feature label Aug 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category:ETL Extract Transform Load new feature New functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants