#3260 - CAS Integration 3A - Create new Supplier and Site - Retry on Error and Scheduler Time #3830

andrewsignori-aot · 2024-10-24T15:34:04Z

Changed the to allow throwing an error when some error is present in the process summary error attending the below AC.

If some error is present throw in the scheduler to allow it to be retried.

Added the below E2E to test the above change.
- Should throw an error for the first student and process the second one when the CAS API call failed for the first student but worked for the second one.
Adjusted the scheduler to be executed during weekdays only attending the below AC (0 20 * * 1-5).

Schedule these to go once a day at noon during weekdays for now.

Rollback

Fix

COUNTRY_CANADA was defined as "canada" when it is defined as "Canada" is all form.io definitions leading to an incorrect logic.
SFTP issue while providing the file content as a string instead of a Buffer as identified by @dheepak-aot.

sh16011993 · 2024-10-24T16:24:51Z

...ue-consumers/src/processors/schedulers/cas-integration/cas-supplier-integration.scheduler.ts

@@ -63,19 +62,16 @@ export class CASSupplierIntegrationScheduler extends BaseScheduler<void> {
          `Records updated: ${suppliersUpdated}.`,
        ],
        processSummary,
+        { throwOnError: true },


...ue-consumers/src/processors/schedulers/cas-integration/cas-supplier-integration.scheduler.ts

sh16011993

Great work @andrewsignori-aot 👍 Thank You for explaining the mocks related to the e2e test. Very helpful.

dheepak-aot · 2024-10-24T19:42:56Z

sources/packages/backend/apps/db-migrations/src/sql/Queue/Update-cas-supplier-integration.sql

+    sims.queue_configurations
+SET
+    queue_configuration = '{
+        "cron": "0 19 * * 1-5",


AC says noon. Either we can have the AC updated or change here.

My bad. I missed the PT part. As we discussed noon PST should be 8PM UTC.

My bad too. All schedulers on wiki are using PST as a reference.

Update to 8 PM UTC which will convert to 12 PM PST.
Updated the wik also.

...processors/schedulers/cas-integration/_tests_/cas-supplier-integration.scheduler.e2e-spec.ts

dheepak-aot · 2024-10-24T20:04:35Z

sources/packages/backend/libs/utilities/src/string-utils.ts

+export function convertToASCIIString(rawContent?: string): string | null {
+  return convertToASCII(rawContent)?.toString() ?? null;


Thanks for the change.

Thanks for raising the issue 😉

sonarqubecloud · 2024-10-24T20:18:55Z

Quality Gate passed

Issues
0 New issues
2 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-10-24T20:19:49Z

Backend Unit Tests Coverage Report

Totals
Statements:	22.56% ( 3662 / 16233 )
Methods:	10.16% ( 205 / 2018 )
Lines:	25.96% ( 3181 / 12255 )
Branches:	14.08% ( 276 / 1960 )

github-actions · 2024-10-24T20:20:03Z

E2E Workflow Workers Coverage Report

Totals
Statements:	58.64% ( 509 / 868 )
Methods:	52.88% ( 55 / 104 )
Lines:	62.27% ( 411 / 660 )
Branches:	41.35% ( 43 / 104 )

github-actions · 2024-10-24T20:21:57Z

E2E Queue Consumers Coverage Report

Totals
Statements:	83.81% ( 1170 / 1396 )
Methods:	83.94% ( 115 / 137 )
Lines:	84.74% ( 994 / 1173 )
Branches:	70.93% ( 61 / 86 )

github-actions · 2024-10-24T20:36:29Z

E2E SIMS API Coverage Report

Totals
Statements:	66.01% ( 5571 / 8439 )
Methods:	63.3% ( 683 / 1079 )
Lines:	70.12% ( 4401 / 6276 )
Branches:	44.93% ( 487 / 1084 )

dheepak-aot · 2024-10-24T20:45:27Z

sources/packages/backend/apps/db-migrations/src/sql/Queue/Update-cas-supplier-integration.sql

+    sims.queue_configurations
+SET
+    queue_configuration = '{
+        "cron": "0 20 * * 1-5",


Thanks. Please update the PR description as well.

dheepak-aot

Thanks for making the change. Looks good 👍

andrewsignori-aot added 4 commits October 23, 2024 16:33

Adding throw error and Canada fix

0d22c4d

Merge branch 'main' into feature/#3260-cas-integration-part-4

8b9e6f5

Adding E2E test for throw

62a228e

Minor E2E adjustment

f4fe18e

andrewsignori-aot self-assigned this Oct 24, 2024

andrewsignori-aot added the Queue Consumers label Oct 24, 2024

andrewsignori-aot changed the title ~~#3260 - CAS Integration 3A - Create new Supplier and Site - Retry on Error~~ #3260 - CAS Integration 3A - Create new Supplier and Site - Retry on Error and Scheduler Time Oct 24, 2024

DB migrations

b53cb5a

andrewsignori-aot added the DB DB migration involved label Oct 24, 2024

E2E improvement

ffb85e7

andrewsignori-aot marked this pull request as ready for review October 24, 2024 16:10

sh16011993 self-requested a review October 24, 2024 16:13

sh16011993 reviewed Oct 24, 2024

View reviewed changes

dheepak-aot self-requested a review October 24, 2024 16:26

sh16011993 reviewed Oct 24, 2024

View reviewed changes

...ue-consumers/src/processors/schedulers/cas-integration/cas-supplier-integration.scheduler.ts Show resolved Hide resolved

sh16011993 approved these changes Oct 24, 2024

View reviewed changes

Fix SFTP issue

7433a3c

dheepak-aot reviewed Oct 24, 2024

View reviewed changes

...processors/schedulers/cas-integration/_tests_/cas-supplier-integration.scheduler.e2e-spec.ts Show resolved Hide resolved

dheepak-aot reviewed Oct 24, 2024

View reviewed changes

Changed cron expression to PST

2fdbd65

dheepak-aot reviewed Oct 24, 2024

View reviewed changes

dheepak-aot approved these changes Oct 24, 2024

View reviewed changes

andrewsignori-aot added this pull request to the merge queue Oct 24, 2024

Merged via the queue into main with commit 0f8c1b3 Oct 24, 2024
20 checks passed

andrewsignori-aot temporarily deployed to DEV October 24, 2024 20:57 — with GitHub Actions Inactive

andrewsignori-aot temporarily deployed to DEV October 24, 2024 20:58 — with GitHub Actions Inactive

andrewsignori-aot temporarily deployed to DEV October 24, 2024 21:01 — with GitHub Actions Inactive

andrewsignori-aot deleted the feature/#3260-cas-integration-part-4 branch October 28, 2024 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#3260 - CAS Integration 3A - Create new Supplier and Site - Retry on Error and Scheduler Time #3830

#3260 - CAS Integration 3A - Create new Supplier and Site - Retry on Error and Scheduler Time #3830

andrewsignori-aot commented Oct 24, 2024 •

edited

Loading

sh16011993 Oct 24, 2024

sh16011993 left a comment

dheepak-aot Oct 24, 2024

dheepak-aot Oct 24, 2024

andrewsignori-aot Oct 24, 2024

andrewsignori-aot Oct 24, 2024 •

edited

Loading

dheepak-aot Oct 24, 2024

andrewsignori-aot Oct 24, 2024

sonarqubecloud bot commented Oct 24, 2024

github-actions bot commented Oct 24, 2024

github-actions bot commented Oct 24, 2024

github-actions bot commented Oct 24, 2024

github-actions bot commented Oct 24, 2024

dheepak-aot Oct 24, 2024

dheepak-aot left a comment

		export function convertToASCIIString(rawContent?: string): string \| null {
		return convertToASCII(rawContent)?.toString() ?? null;

#3260 - CAS Integration 3A - Create new Supplier and Site - Retry on Error and Scheduler Time #3830

#3260 - CAS Integration 3A - Create new Supplier and Site - Retry on Error and Scheduler Time #3830

Conversation

andrewsignori-aot commented Oct 24, 2024 • edited Loading

Rollback

Fix

sh16011993 Oct 24, 2024

Choose a reason for hiding this comment

sh16011993 left a comment

Choose a reason for hiding this comment

dheepak-aot Oct 24, 2024

Choose a reason for hiding this comment

dheepak-aot Oct 24, 2024

Choose a reason for hiding this comment

andrewsignori-aot Oct 24, 2024

Choose a reason for hiding this comment

andrewsignori-aot Oct 24, 2024 • edited Loading

Choose a reason for hiding this comment

dheepak-aot Oct 24, 2024

Choose a reason for hiding this comment

andrewsignori-aot Oct 24, 2024

Choose a reason for hiding this comment

sonarqubecloud bot commented Oct 24, 2024

Quality Gate passed

github-actions bot commented Oct 24, 2024

Backend Unit Tests Coverage Report

github-actions bot commented Oct 24, 2024

E2E Workflow Workers Coverage Report

github-actions bot commented Oct 24, 2024

E2E Queue Consumers Coverage Report

github-actions bot commented Oct 24, 2024

E2E SIMS API Coverage Report

dheepak-aot Oct 24, 2024

Choose a reason for hiding this comment

dheepak-aot left a comment

Choose a reason for hiding this comment

andrewsignori-aot commented Oct 24, 2024 •

edited

Loading

andrewsignori-aot Oct 24, 2024 •

edited

Loading