#3260 - CAS Integration 3A - Create new Supplier and Site - CAS API Call #3804

andrewsignori-aot · 2024-10-17T23:12:33Z

Main Changes

The current process is now segregated into different classes (called processors) that are responsible for taking care of the below scenarios.
- CASActiveSupplierNotFoundProcessor: 🆕
  - Supplier does not exist on CAS;
  - CAS API should be called to create supplier and site;
  - Student CAS supplier should be updated.
- CASPreValidationsProcessor: 🆕
  - Given name or not a Canadian address.
  - Update the student supplier for manual intervention.
- CASActiveSupplierFoundProcessor: (🆕 refactored)
  - Supplier found on CAS
  - Student CAS supplier should be updated.

Address match

Included the address match logic in the evaluation processes following the logic already decided in CAS Integration 3B - Create New Site (Supplier found, Multiple Sites / No matching address) #3729. If an active address with a valid match is found it will be persisted and the CAS supplier will be considered valid. If the address does not exist or does not match, the supplier will be saved but the CAS record will be considered invalid.

Error handling

API call properly reports errors using HTTP Status 400 and adding meaningful error messages. Please see below samples.
These error messages were used to define the max length validation for the submitted data.

[0069] Length of field SupplierName exceeded. Maximum length is 80.
[0034] SIN is already in use.
[0069] Length of field AddressLine1 exceeded. Maximum length is 35.

GetSupplier 404

The GetSupplier method from CAS API does not return a 404 error when no supplier is found as stated in the documentation.
The method seems to be implemented as a search and it will just return an empty array if no items were found.

{
    "items": [],
    "hasMore": false,
    "limit": 0,
    "offset": 0,
    "count": 0,
}

Secondary PR changes

Created a small helper to identify if the student's address is from Canada. The idea is to avoid confusion about which property to use and how the value is set on DB. This is the reason the method receives the AddressInfo itself which contains country and selectedCountry properties.
New CAS mocks added to allow the CAS responses mocks.

New E2E Tests

Should update CAS supplier table to manual intervention when student does not have a first name.
Should update CAS supplier table to manual intervention when student address is not from Canada.
Should create a new supplier and site on CAS and update CAS suppliers table when the student was not found on CAS and the request to create the supplier and site was successful.

Log samples

[2024-10-17 22:53:31]--INFO: Log details
[2024-10-17 22:53:31]----INFO: Log details
[2024-10-17 22:53:31]----INFO: Logging on CAS.
[2024-10-17 22:53:31]----INFO: Logon successful.
[2024-10-17 22:53:31]------INFO: Log details
[2024-10-17 22:53:31]------INFO: Processing student CAS supplier ID: 69.
[2024-10-17 22:53:31]------INFO: CAS evaluation result status: NotFound.
[2024-10-17 22:53:31]------INFO: No active CAS supplier found. Reason: Supplier not found.
[2024-10-17 22:53:31]------INFO: Created supplier and site on CAS.
[2024-10-17 22:53:31]------INFO: Updated CAS supplier and site for the student.
[2024-10-17 22:53:31]INFO: CAS supplier integration executed.

[2024-10-17 23:04:38]INFO: Log details
[2024-10-17 23:04:38]INFO: Executing CAS supplier integration...
[2024-10-17 23:04:38]INFO: Found 1 records to be updated.
[2024-10-17 23:04:38]--INFO: Log details
[2024-10-17 23:04:38]----INFO: Log details
[2024-10-17 23:04:38]----INFO: Logging on CAS.
[2024-10-17 23:04:38]----INFO: Logon successful.
[2024-10-17 23:04:38]------INFO: Log details
[2024-10-17 23:04:38]------INFO: Processing student CAS supplier ID: 86.
[2024-10-17 23:04:38]------INFO: CAS evaluation result status: PreValidationsFailed.
[2024-10-17 23:04:38]------WARN: Not possible to retrieve CAS supplier information because some pre-validations were not fulfilled. Reason(s): Non-Canadian address.
[2024-10-17 23:04:38]------INFO: Updated CAS supplier for manual intervention.
[2024-10-17 23:04:38]INFO: CAS supplier integration executed.

[2024-10-17 22:55:57]INFO: Log details
[2024-10-17 22:55:57]INFO: Executing CAS supplier integration...
[2024-10-17 22:55:57]INFO: Found 1 records to be updated.
[2024-10-17 22:55:57]--INFO: Log details
[2024-10-17 22:55:57]----INFO: Log details
[2024-10-17 22:55:57]----INFO: Logging on CAS.
[2024-10-17 22:55:57]----INFO: Logon successful.
[2024-10-17 22:55:57]------INFO: Log details
[2024-10-17 22:55:57]------INFO: Processing student CAS supplier ID: 74.
[2024-10-17 22:55:57]------INFO: CAS evaluation result status: ActiveSupplierFound.
[2024-10-17 22:55:57]------INFO: Active CAS supplier found.
[2024-10-17 22:55:57]------INFO: Updated CAS supplier for the student.
[2024-10-17 22:55:57]INFO: CAS supplier integration executed.

TODO

Adjust the scheduler cron expression.
Add more E2E tests.
Run in parallel.
If some error is present throw in the scheduler to allow it to be retried.

andrewsignori-aot · 2024-10-18T20:04:48Z

sources/packages/backend/libs/integrations/src/cas/models/cas-service.model.ts

Changes related to this PR start at line 50.
The file was renamed.

sources/packages/backend/libs/integrations/src/cas/cas-formaters.ts

sources/packages/backend/libs/utilities/src/address-utils.ts

sources/packages/backend/libs/integrations/src/cas/models/cas-service.model.ts

sources/packages/backend/libs/utilities/src/address-utils.ts

sh16011993 · 2024-10-18T23:09:08Z

sources/packages/backend/libs/integrations/src/cas/cas-formatters.ts

sources/packages/backend/apps/queue-consumers/src/services/cas-supplier/cas-supplier.service.ts

dheepak-aot · 2024-10-21T16:26:29Z

sources/packages/backend/apps/queue-consumers/src/services/cas-supplier/cas-supplier.service.ts

      const auth = await this.casService.logon();
      summary.info("Logon successful.");
-      suppliersUpdated = await this.requestCASAndUpdateSuppliers(
+      suppliersUpdated = await this.processSuppliers(


The silent failing is also happening inside this.processSuppliers.

Does this method need to fail silently w/o throwing error?

The below AC is not implemented yet.
"If some error is present throw in the scheduler to allow it to be retried."
The idea is to inspect the summary (as it is already done in getSuccessMessageWithAttentionCheck) and throw an error if any error log message is present. Throwing an exception from here or from there would result in the same.
I added it now to the "TODOs" in the PR description.

I think the idea is that the failure will be caught and then logged for info purpose as summary.error and then in the next run of the scheduler, the failed cas suppliers will be attempted again.

If some error is logged to the summary we will throw to allow the scheduler to retry immediately.
When we have the permanent errors implemented, those should be probably logged as warnings because they will be part of the "business valid scenarios".

dheepak-aot · 2024-10-21T16:47:01Z

sources/packages/backend/apps/queue-consumers/src/services/cas-supplier/cas-supplier.service.ts

@@ -38,10 +50,10 @@ export class CASSupplierIntegrationService {
    const summary = new ProcessSummary();
    parentProcessSummary.children(summary);
    try {
-      summary.info("Logging on CAS...");
+      summary.info("Logging on CAS.");
      const auth = await this.casService.logon();


Not related to PR and asking for my understanding, How long is a logon valid?

As per the docs, 1 hour (3600 seconds).

...services/cas-supplier/cas-evaluation-result-processor/cas-active-supplier-found-processor.ts

...s/src/services/cas-supplier/cas-evaluation-result-processor/cas-pre-validations-processor.ts

dheepak-aot · 2024-10-21T17:54:49Z

...ices/cas-supplier/cas-evaluation-result-processor/cas-active-supplier-not-found-processor.ts

+        {
+          supplierNumber: result.response.supplierNumber,
+          supplierName: result.submittedData.SupplierName,
+          status: "ACTIVE",
+          lastUpdated: now,
+          supplierAddress: {
+            supplierSiteCode: result.response.supplierSiteCode,
+            addressLine1: submittedAddress.AddressLine1,
+            city: submittedAddress.City,
+            provinceState: submittedAddress.Province,
+            country: submittedAddress.Country,
+            postalCode: submittedAddress.PostalCode,
+            status: "ACTIVE",
+            lastUpdated: now,
+          },


what about supplierProtected and siteProtected in the update?

Do you mean setting them explicitly to null?
Since the supplier was not found, it will be a new supplier and site. These properties should never exist or be created for a new supplier and site.

These properties should never exist or be created for a new supplier and site. - Ah, is it? I was under the impression that it is set during new supplier and site creation.

They are legacy properties. As far as I know, they only exist due to some SFAS requirement.
Either way, we do not have any direction about setting them.

dheepak-aot · 2024-10-21T17:57:21Z

...ices/cas-supplier/cas-evaluation-result-processor/cas-active-supplier-not-found-processor.ts

+        {
+          id: casSupplier.id,
+        },
+        {


The update happening for found and not found is identical. Can we have a common method to be re-used for the update?

Do you mean the below part is identical? I would prefer to leave as it is.

My point was conversion of CAS Supplier and CAS Site to SIMS Supplier and SIMS supplier address(and then update). May be something in one of the upcoming CAS tickets.

sh16011993 · 2024-10-21T18:05:19Z

...ices/cas-supplier/cas-evaluation-result-processor/cas-active-supplier-not-found-processor.ts

+    if (evaluationResult.status !== CASEvaluationStatus.NotFound) {
+      throw new Error("Incorrect CAS evaluation result processor selected.");
+    }
+    summary.info(
+      `No active CAS supplier found. Reason: ${evaluationResult.reason}.`,
+    );


👍 for this info in all the processors.

dheepak-aot · 2024-10-21T18:38:30Z

Great Job @andrewsignori-aot. Thanks for trying it make it streamlined for any upcoming CAS updates.
Please have a look at the comments.

sources/packages/backend/libs/integrations/src/cas/cas.service.ts

...services/cas-supplier/cas-evaluation-result-processor/cas-active-supplier-found-processor.ts

sh16011993 · 2024-10-21T19:36:04Z

...processors/schedulers/cas-integration/_tests_/cas-supplier-integration.scheduler.e2e-spec.ts

+      modifier: { id: systemUsersService.systemUser.id },
+    });
+    // Ensure updatedAt was updated.
+    expect(updateCASSupplier.updatedAt.getTime()).toBeGreaterThan(


sh16011993 · 2024-10-21T19:39:07Z

...processors/schedulers/cas-integration/_tests_/cas-supplier-integration.scheduler.e2e-spec.ts

+      // Arrange
+      const referenceDate = new Date();
+      const savedCASSupplier = await saveFakeCASSupplier(db);
+      // Configure CAS mock to return em empty result for the GetSupplier


Minor: an

sh16011993

Great work @andrewsignori-aot 👍 Thank You for the walkthrough of your PR.

sonarqubecloud · 2024-10-21T21:34:46Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-10-21T21:35:32Z

Backend Unit Tests Coverage Report

Totals
Statements:	22.42% ( 3632 / 16201 )
Methods:	9.99% ( 201 / 2013 )
Lines:	25.8% ( 3158 / 12238 )
Branches:	14% ( 273 / 1950 )

github-actions · 2024-10-21T21:35:57Z

E2E Workflow Workers Coverage Report

Totals
Statements:	58.64% ( 509 / 868 )
Methods:	52.88% ( 55 / 104 )
Lines:	62.27% ( 411 / 660 )
Branches:	41.35% ( 43 / 104 )

github-actions · 2024-10-21T21:37:57Z

E2E Queue Consumers Coverage Report

Totals
Statements:	83.21% ( 1160 / 1394 )
Methods:	83.94% ( 115 / 137 )
Lines:	84.26% ( 990 / 1175 )
Branches:	67.07% ( 55 / 82 )

github-actions · 2024-10-21T21:40:44Z

E2E SIMS API Coverage Report

Totals
Statements:	66.01% ( 5571 / 8439 )
Methods:	63.3% ( 683 / 1079 )
Lines:	70.12% ( 4401 / 6276 )
Branches:	44.93% ( 487 / 1084 )

dheepak-aot

Great work. Thanks for doing the changes. 👍

andrewsignori-aot added 6 commits October 11, 2024 15:57

Work in progress for supplier evaluation.

177068b

Created models for CAS evaluation.

e55ba29

WIP calling API to create supplier.

6a35a93

Creating supplier and site.

38dd692

Create processors

7828588

E2E tests passing with mocked CAS service.

ee79c8d

andrewsignori-aot added Queue Consumers Integration labels Oct 17, 2024

andrewsignori-aot self-assigned this Oct 17, 2024

andrewsignori-aot added 6 commits October 17, 2024 16:16

Fix import for country.

d14b1ae

Imports adjustment.

c7e4bc5

Address match and E2E fix

4f77728

Added city max length validation.

209a976

Code improvement.

8e080bf

Fix E2E to ensure address match

caf5e44

andrewsignori-aot commented Oct 18, 2024

View reviewed changes

E2E adjustment

4afa340

andrewsignori-aot marked this pull request as ready for review October 18, 2024 20:06