Skip to content

Check that a PR driver is still running before trying to kill it#2799

Merged
WalterKolczynski-NOAA merged 8 commits into
NOAA-EMC:developfrom
DavidHuber-NOAA:fix/bash_ci
Aug 7, 2024
Merged

Check that a PR driver is still running before trying to kill it#2799
WalterKolczynski-NOAA merged 8 commits into
NOAA-EMC:developfrom
DavidHuber-NOAA:fix/bash_ci

Conversation

@DavidHuber-NOAA
Copy link
Copy Markdown
Contributor

@DavidHuber-NOAA DavidHuber-NOAA commented Jul 30, 2024

Description

Adds a check to the SSH command used to kill child PIDs of a defunct driver instance on a different head node to prevent invalid kill commands, preventing CI failures.

Resolves #2798

Type of change

  • Bug fix (fixes something broken)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO

How has this been tested?

Successfully detected the driver was no longer running on WCOSS2 for PR #2791.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes

Comment thread ci/scripts/driver.sh Fixed
Comment thread ci/scripts/driver.sh Fixed
@DavidHuber-NOAA DavidHuber-NOAA changed the title Make PR dir if missing; check that driver is running before killing Check that a PR driver is still running before trying to kill it Jul 30, 2024
Copy link
Copy Markdown
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but hold for @TerrenceMcGuinness-NOAA to get back and review it.

@WalterKolczynski-NOAA
Copy link
Copy Markdown
Contributor

@TerrenceMcGuinness-NOAA Please review this so it can move forward.

Comment thread ci/scripts/check_ci.sh
Comment thread ci/scripts/driver.sh Outdated
Copy link
Copy Markdown
Collaborator

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice updates.

@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit ad8d3e9 into NOAA-EMC:develop Aug 7, 2024
CoryMartin-NOAA pushed a commit to CoryMartin-NOAA/global-workflow that referenced this pull request Aug 7, 2024
…A-EMC#2799)

Adds a check to the SSH command used to kill child PIDs of a defunct
driver instance on a different head node to prevent invalid kill
commands, preventing CI failures.

Resolves NOAA-EMC#2798
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Aug 9, 2024
* origin/develop:
  Stage atmospheric backgrounds and UFS cubed-sphere history files (NOAA-EMC#2792)
  Check that a PR driver is still running before trying to kill it (NOAA-EMC#2799)
  Feature/get arch adds an empty archive job to GEFS system (NOAA-EMC#2772)
  Marine DA updates (NOAA-EMC#2802)
  Revert MSU FIX_DIRs back to glopara (NOAA-EMC#2811)
  Bugfix for updating label states in Jenkins (NOAA-EMC#2808)
  Clean-up temporary rundirs - take 2. (NOAA-EMC#2753)
  Change land surface for HR4 (NOAA-EMC#2787)
  Run METplus serially and correct the name of prod tasks (NOAA-EMC#2804)
  Update Java Agent launching script for Jenkins connections (NOAA-EMC#2762)
  Fix erroneous cdump addition (NOAA-EMC#2803)
  Update ocean post-processing triggers (NOAA-EMC#2784)
  Update the gfs_utils repository hash (NOAA-EMC#2801)
  Add fixes for metplus jobs when gfs_cyc=2 or 4 (NOAA-EMC#2791)
  Simplify resource-related variables, remove CDUMP where unneeded (NOAA-EMC#2727)
  Remove f000 from atmos rocoto tasks for replay cases (NOAA-EMC#2778)
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this pull request Aug 9, 2024
* origin/develop:
  Stage atmospheric backgrounds and UFS cubed-sphere history files (NOAA-EMC#2792)
  Check that a PR driver is still running before trying to kill it (NOAA-EMC#2799)
  Feature/get arch adds an empty archive job to GEFS system (NOAA-EMC#2772)
  Marine DA updates (NOAA-EMC#2802)
  Revert MSU FIX_DIRs back to glopara (NOAA-EMC#2811)
  Bugfix for updating label states in Jenkins (NOAA-EMC#2808)
  Clean-up temporary rundirs - take 2. (NOAA-EMC#2753)
  Change land surface for HR4 (NOAA-EMC#2787)
  Run METplus serially and correct the name of prod tasks (NOAA-EMC#2804)
  Update Java Agent launching script for Jenkins connections (NOAA-EMC#2762)
  Fix erroneous cdump addition (NOAA-EMC#2803)
  Update ocean post-processing triggers (NOAA-EMC#2784)
  Update the gfs_utils repository hash (NOAA-EMC#2801)
  Add fixes for metplus jobs when gfs_cyc=2 or 4 (NOAA-EMC#2791)
  Simplify resource-related variables, remove CDUMP where unneeded (NOAA-EMC#2727)
  Remove f000 from atmos rocoto tasks for replay cases (NOAA-EMC#2778)
@DavidHuber-NOAA DavidHuber-NOAA deleted the fix/bash_ci branch August 13, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bash CI attempts to kill non-existent driver instances

4 participants