Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOFAST watchdog process #69

Open
bbean23 opened this issue Apr 3, 2024 · 0 comments
Open

SOFAST watchdog process #69

bbean23 opened this issue Apr 3, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@bbean23
Copy link
Collaborator

bbean23 commented Apr 3, 2024

Add a new process that can restart SOFAST in the case that it crashes unexpectedly. We believe that SOFAST should be stable, but this should be available for high-availability use cases as a back up.

Requirements of the watchdog process:

  • very simple to improve stability
  • minimal imports
  • occasionally make a request to SOFAST to check for responsiveness
  • restart SOFAST when it takes more than a threshold length of seconds to respond to requests
  • restart SOFAST when system memory usage reaches a threshold
  • create alert files when system memory usage reaches a critical threshold for an extended period of time
  • create alert files when SOFAST is taking longer to respond than expected, but not long enough to restart it
  • create alert files when SOFAST is restarted withing a threshold of seconds after restarting
  • customizable threshold levels determined from a settings file
@bbean23 bbean23 added the enhancement New feature or request label Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant