terraform/module: Avoid running a job if nothing has changed

[radeksimko]

Background

This is a different approach to addressing a performance bottleneck when we have duplicate jobs, as originally described in #989

Proposal

Remove deduplication logic from the scheduler:

terraform-ls/internal/state/jobs.go

Lines 61 to 75 in ca335f5

	jobID, queued, err := js.jobExists(newJob, StateQueued)
	if err != nil {
	return "", err
	}
	if queued {
	return jobID, nil
	}
	jobID, running, err := js.jobExists(newJob, StateRunning)
	if err != nil {
	return "", err
	}
	if running {
	return jobID, nil
	}

Introduce IgnoreExistingState bool to job.Job{}, to indicate whether to ignore existing data/cache.

• initialize - IgnoreExistingState: false
• textDocument/didOpen - IgnoreExistingState: true - this is only necessary to account for an edge case where the user opens editor with unsaved files from previous session, we could potentially avoid it if we tracked ModTime or any kind of indicator allowing us to compare the file being opened with the one already indexed, effectively implementing this TODO
• textDocument/didChange - IgnoreExistingState: true
• workspace/didChangeWorkspaceFolders - IgnoreExistingState: true
• workspace/didChangeWatchedFiles - IgnoreExistingState: true

The job could return return an error, such as NotChangedError{Handle: document.DirHandle}, such that downstream jobs can decide based on that, whether to run.

• GetTerraformVersion - decoder: Pick core schema only based on required_version constraint #993
• ObtainSchema - terraform/module: Introduce new job to parse provider versions from lock file #992
• ParseModuleConfiguration - check ModuleParsingState
• ParseVariables - check VarsParsingState
• ParseModuleManifest - check ModManifestState
• LoadModuleMetadata - check MetaState field + NotChangedError
• DecodeReferenceTargets - check NotChangedError
• DecodeReferenceOrigins - check NotChangedError
• DecodeVarsReferences - check NotChangedError
• GetModuleDataFromRegistry - check NotChangedError

Implementation Notes

Regarding NotChangedError - some downstream jobs run after multiple different jobs, so we need to make sure that the upstream job actually returns NotChangedError if and only if all jobs detect no changes.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.