#3260 - CAS Integration 3A - Create new Supplier and Site - Process in Parallel #3837
Conversation
andrewsignori-aot
commented
Oct 24, 2024
•
andrewsignori-aot
commented
- Adjusted supplier's processes to happen in parallel.
- Reduced the amount of allowed parallel processes to 2 to 4. Previously it was using the "os cores" as the default value which could potentially be up to 32 while running on Openshift.
This reverts commit 3c62c42.
" Increased the amount of allowed parallel processes from 2 to 4." |
| /** | ||
| * Number of parallel processes allowed to be started at same time. | ||
| */ | ||
| export enum ParallelIntensity { |
| @@ -48,6 +54,9 @@ export class CASSupplierIntegrationService { | |||
| parentProcessSummary: ProcessSummary, | |||
| studentSuppliers: StudentSupplierToProcess[], | |||
| ): Promise<number> { | |||
| // Force the CAS token to be acquired before starting the process. | |||
| await this.casService.getToken(); | |||
There was a problem hiding this comment.
Great. When the get token fails, process is aborted right away.
My comment was not clear, I meant that the default value was decreased from the "os.cores" amount (potentially 32 on Openshift) to 2 or 4 😉 |
sources/packages/backend/libs/utilities/src/parallel-processing-utils.ts
Show resolved
Hide resolved
| @@ -48,6 +54,9 @@ export class CASSupplierIntegrationService { | |||
| parentProcessSummary: ProcessSummary, | |||
| studentSuppliers: StudentSupplierToProcess[], | |||
| ): Promise<number> { | |||
| // Force the CAS token to be acquired before starting the process. | |||
| await this.casService.getToken(); | |||
There was a problem hiding this comment.
Minor and not a blocker, the method we are looking here is like initToken right?
There was a problem hiding this comment.
This is the same method used internally by the method to get a token (request one or cached one).
There is no guarantee that this method will be the one initializing the token, for instance, if the scheduler is retrying due to some individual student error the token will still be cached.
The method getToken can be kept private (as it was before) and we can create something like ensureTokenInitialization.
I reused the same method for simplicity. Please let me know if it would be a blocker or if it requires a quick chat 😉
|
Thank you very much for making the change. minor comments and a question. |
|
Backend Unit Tests Coverage Report
|
E2E Workflow Workers Coverage Report
|
E2E Queue Consumers Coverage Report
|
E2E SIMS API Coverage Report
|
