Option to limit run time of VM

[jglick]

What feature do you want to see added?

CleanLostNodesWork tries to terminate VMs that no longer seem to be associated with any known Jenkins agent, but this still relies on the Jenkins controller to perform cleanup.

google-compute-engine-plugin/src/main/java/com/google/jenkins/plugins/computeengine/ComputeEngineCloud.java

Lines 176 to 177 in 0157d4f

	* <p>This ID allows us to find machines from our cloud in GCP. <b>This value should not change
	* between config reload, or nodes may be lost in GCP side</b>

notes that this may not be foolproof; in particular, if you are using the configuration-as-code plugin to define a list of clouds, you may well neglect to set instanceId, in which case the id would be randomly reset every time JCasC was reapplied.

It would be useful to have an option to set .scheduling.maxRunDuration to ensure that an instance is automatically terminated if it has been accidentally left running for an unreasonably long time.

Even better, set .scheduling.terminationTime to a day in the future when the instance is created, and then replace CleanLostNodesWork with an hourly task to look for all connected GCE agents and update their termination time to be a day from then. This would guarantee that any abandoned VM will be terminated in a timely fashion no matter what happens to Jenkins, without imposing a particular time limit on how long a VM can legitimately be used as an agent if you happen to have an especially long build.

Upstream changes

I checked

google-compute-engine-plugin/src/main/java/com/google/jenkins/plugins/computeengine/InstanceConfiguration.java

Lines 497 to 501 in 0157d4f

	private Scheduling scheduling() {
	Scheduling scheduling = new Scheduling();
	scheduling.setPreemptible(preemptible);
	return scheduling;
	}

but the current client library seems to lack support for this system. Perhaps it is too old? There seem to be multiple Java libraries that have been published by Google and it is not clear which one is preferred and supported. Recommend updating the client library version here and in dependencies such as google-oauth-plugin.

[jglick]

See jenkinsci/kubernetes-plugin#1543 for an example of a similar feature.