Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1864 - MSFAA Receive File E2E Test Automation #1913

Merged
merged 7 commits into from
May 2, 2023
Merged

#1864 - MSFAA Receive File E2E Test Automation #1913

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a02b906
Mock file download
andrewsignori-aot Apr 28, 2023
a4bb3ca
MSFAA receive file E2E
andrewsignori-aot Apr 28, 2023
228c435
Fix processResult assert
andrewsignori-aot Apr 28, 2023
511319d
Created methods to change file content
andrewsignori-aot Apr 28, 2023
2bf796b
Pre review adjustments.
andrewsignori-aot May 2, 2023
d94eda7
PR reviews fixes.
andrewsignori-aot May 2, 2023
2ea04ea
canceled to cancelled
andrewsignori-aot May 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
325 changes: 325 additions & 0 deletions ...aa-integration/_tests_/msfaa-part-time-process-response-integration.scheduler.e2e-spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,325 @@
import { DeepMocked, createMock } from "@golevelup/ts-jest";
import { INestApplication } from "@nestjs/common";
import { QueueNames, getISODateOnlyString } from "@sims/utilities";
import {
createTestingAppModule,
describeProcessorRootTest,
} from "../../../../../../test/helpers";
import { PartTimeMSFAAProcessResponseIntegrationScheduler } from "../msfaa-part-time-process-response-integration.scheduler";
import {
E2EDataSources,
createE2EDataSources,
createFileFromStructuredRecords,
getStructuredRecords,
mockDownloadFiles,
MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD,
MSFAA_PART_TIME_RECEIVE_FILE_WITH_INVALID_RECORDS_COUNT,
MSFAA_PART_TIME_RECEIVE_FILE_WITH_INVALID_SIN_HASH_TOTAL,
} from "@sims/test-utils";
import * as Client from "ssh2-sftp-client";
import { Job } from "bull";
import * as path from "path";
import {
MSFAA_PART_TIME_MARRIED,
MSFAA_PART_TIME_OTHER_COUNTRY,
MSFAA_PART_TIME_RELATIONSHIP_OTHER,
} from "./msfaa-part-time-process-integration.scheduler.models";
import { saveMSFAATestInputsData } from "./msfaa-factory";
import { In, IsNull } from "typeorm";

describe(
describeProcessorRootTest(QueueNames.PartTimeMSFAAProcessResponseIntegration),
() => {
let app: INestApplication;
let processor: PartTimeMSFAAProcessResponseIntegrationScheduler;
let db: E2EDataSources;
let sftpClientMock: DeepMocked<Client>;
let msfaaMocksDownloadFolder: string;

beforeAll(async () => {
msfaaMocksDownloadFolder = path.join(__dirname, "msfaa-receive-files");
// Set the ESDC response folder to the files mocks folders.
process.env.ESDC_RESPONSE_FOLDER = msfaaMocksDownloadFolder;
const { nestApplication, dataSource, sshClientMock } =
await createTestingAppModule();
app = nestApplication;
db = createE2EDataSources(dataSource);
sftpClientMock = sshClientMock;
// Processor under test.
processor = app.get(PartTimeMSFAAProcessResponseIntegrationScheduler);
});

beforeEach(async () => {
jest.clearAllMocks();
// Force any not signed MSFAA to be signed to ensure that new
// ones created will be the only ones available to be updated.
await db.msfaaNumber.update(
{ dateSigned: IsNull() },
{ dateSigned: getISODateOnlyString(new Date()) },
);
// Cancel any pending MSFAA.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mm...here what does it mean "pending MSFAA"? when dateSigned === null then MSFAA is pending, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

await db.msfaaNumber.update(
{ cancelledDate: IsNull() },
{ cancelledDate: getISODateOnlyString(new Date()) },
);
});

it("Should process an MSFAA response with confirmations and a cancellation and update all records when the file is received as expected.", async () => {
// Arrange
const msfaaInputData = [
MSFAA_PART_TIME_MARRIED,
MSFAA_PART_TIME_OTHER_COUNTRY,
MSFAA_PART_TIME_RELATIONSHIP_OTHER,
];
const createdMSFAARecords = await saveMSFAATestInputsData(
db,
msfaaInputData,
);

// Queued job.
const job = createMock<Job<void>>();

mockDownloadFiles(sftpClientMock, [
MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD,
]);

// Act
const processResult = await processor.processMSFAAResponses(job);

// Assert
expect(processResult).toStrictEqual([
{
processSummary: [
`Processing file ${MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD}.`,
"File contains:",
"Confirmed MSFAA records (type R): 2.",
"Cancelled MSFAA records (type C): 1.",
"Record from line 1, updated as confirmed.",
"Record from line 3, updated as confirmed.",
"Record from line 2, updated as canceled.",
],
errorsSummary: [],
},
]);
// Assert that the file was deleted from SFTP.
expect(sftpClientMock.delete).toHaveBeenCalled();
// Find the updated MSFAA records previously created.
const msfaaIDs = createdMSFAARecords.map((msfaa) => msfaa.id);
const msfaaUpdatedRecords = await db.msfaaNumber.find({
select: {
msfaaNumber: true,
dateSigned: true,
serviceProviderReceivedDate: true,
cancelledDate: true,
newIssuingProvince: true,
},
where: {
id: In(msfaaIDs),
},
order: {
msfaaNumber: "ASC",
},
});
expect(msfaaUpdatedRecords).toHaveLength(msfaaInputData.length);
const [fistSignedMSFAA, cancelledMSFAA, secondSignedMSFAA] =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the name of the variable should be "firstSignedMSFAA".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

msfaaUpdatedRecords;
// Validate fist confirmed record.
expect(fistSignedMSFAA.dateSigned).toBe("2021-11-20");
expect(fistSignedMSFAA.serviceProviderReceivedDate).toBe("2021-11-21");
// Validate cancelled record.
expect(cancelledMSFAA.cancelledDate).toBe("2021-11-24");
expect(cancelledMSFAA.newIssuingProvince).toBe("ON");
// Validate second confirmed record.
expect(secondSignedMSFAA.dateSigned).toBe("2021-11-22");
expect(secondSignedMSFAA.serviceProviderReceivedDate).toBe("2021-11-23");
});

it("Should successfully process 2 MSFAA records when a file has 3 records but one throws an error during DB update.", async () => {
// Arrange
// Crate only 2 records instead of 3 to force an error while updating the missing record.
const msfaaInputData = [
MSFAA_PART_TIME_OTHER_COUNTRY,
MSFAA_PART_TIME_RELATIONSHIP_OTHER,
];
const createdMSFAARecords = await saveMSFAATestInputsData(
db,
msfaaInputData,
);

// Queued job.
const job = createMock<Job<void>>();

mockDownloadFiles(sftpClientMock, [
MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD,
]);

// Act
const processResults = await processor.processMSFAAResponses(job);

// Assert
const expectedFilePath = `${msfaaMocksDownloadFolder}/${MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD}`;
expect(processResults).toStrictEqual([
{
processSummary: [
`Processing file ${MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD}.`,
"File contains:",
"Confirmed MSFAA records (type R): 2.",
"Cancelled MSFAA records (type C): 1.",
"Record from line 3, updated as confirmed.",
"Record from line 2, updated as canceled.",
],
errorsSummary: [
`Error processing record line number 1 from file ${expectedFilePath}`,
],
},
]);
// Assert that the file was deleted from SFTP.
expect(sftpClientMock.delete).toHaveBeenCalled();
// Find the updated MSFAA records previously created.
const msfaaIDs = createdMSFAARecords.map((msfaa) => msfaa.id);
const msfaaUpdatedRecords = await db.msfaaNumber.find({
select: {
msfaaNumber: true,
dateSigned: true,
serviceProviderReceivedDate: true,
cancelledDate: true,
newIssuingProvince: true,
},
where: {
id: In(msfaaIDs),
},
order: {
msfaaNumber: "ASC",
},
});
expect(msfaaUpdatedRecords).toHaveLength(msfaaInputData.length);
const [cancelledMSFAA, secondSignedMSFAA] = msfaaUpdatedRecords;
// Validate cancelled record.
expect(cancelledMSFAA.cancelledDate).toBe("2021-11-24");
expect(cancelledMSFAA.newIssuingProvince).toBe("ON");
// Validate second confirmed record.
expect(secondSignedMSFAA.dateSigned).toBe("2021-11-22");
expect(secondSignedMSFAA.serviceProviderReceivedDate).toBe("2021-11-23");
});

it("Should throw an error when the MSFAA file contains an invalid SIN hash total.", async () => {
// Arrange
// Queued job.
const job = createMock<Job<void>>();

mockDownloadFiles(sftpClientMock, [
MSFAA_PART_TIME_RECEIVE_FILE_WITH_INVALID_SIN_HASH_TOTAL,
]);

// Act
const processResult = await processor.processMSFAAResponses(job);

// Assert
expect(processResult).toStrictEqual([
{
processSummary: [
`Processing file ${MSFAA_PART_TIME_RECEIVE_FILE_WITH_INVALID_SIN_HASH_TOTAL}.`,
],
errorsSummary: [
"Error downloading file msfaa-part-time-receive-file-with-invalid-sin-hash-total.dat. Error: The MSFAA file has TotalSINHash inconsistent with the total sum of sin in the records",
],
},
]);
// Assert that the file was not deleted from SFTP.
expect(sftpClientMock.delete).not.toHaveBeenCalled();
});

it("Should throw an error when the MSFAA file contains an invalid record count.", async () => {
// Arrange
// Queued job.
const job = createMock<Job<void>>();

mockDownloadFiles(sftpClientMock, [
MSFAA_PART_TIME_RECEIVE_FILE_WITH_INVALID_RECORDS_COUNT,
]);

// Act
const processResult = await processor.processMSFAAResponses(job);

// Assert
expect(processResult).toStrictEqual([
{
processSummary: [
`Processing file ${MSFAA_PART_TIME_RECEIVE_FILE_WITH_INVALID_RECORDS_COUNT}.`,
],
errorsSummary: [
"Error downloading file msfaa-part-time-receive-file-with-invalid-records-count.dat. Error: The MSFAA file has invalid number of records",
],
},
]);
// Assert that the file was not deleted from SFTP.
expect(sftpClientMock.delete).not.toHaveBeenCalled();
});

it("Should throw an error when the MSFAA file contains an invalid header code.", async () => {
// Arrange
// Queued job.
const job = createMock<Job<void>>();

mockDownloadFiles(
sftpClientMock,
[MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD],
(fileContent: string) => {
// Force the header to be wrong.
return fileContent.replace("100", "999");
},
);

// Act
const processResult = await processor.processMSFAAResponses(job);

// Assert
expect(processResult).toStrictEqual([
{
processSummary: [
`Processing file ${MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD}.`,
],
errorsSummary: [
"Error downloading file msfaa-part-time-receive-file-with-cancelation-record.dat. Error: The MSFAA file has an invalid transaction code on header",
],
},
]);
// Assert that the file was not deleted from SFTP.
expect(sftpClientMock.delete).not.toHaveBeenCalled();
});

it("Should throw an error when the MSFAA file contains an invalid footer code.", async () => {
// Arrange
// Queued job.
const job = createMock<Job<void>>();

mockDownloadFiles(
sftpClientMock,
[MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD],
(fileContent: string) => {
const file = getStructuredRecords(fileContent);
// Force the footer to be wrong.
file.footer = file.footer.replace("999", "001");
return createFileFromStructuredRecords(file);
},
);

// Act
const processResult = await processor.processMSFAAResponses(job);

// Assert
expect(processResult).toStrictEqual([
{
processSummary: [
`Processing file ${MSFAA_PART_TIME_RECEIVE_FILE_WITH_CANCELATION_RECORD}.`,
],
errorsSummary: [
"Error downloading file msfaa-part-time-receive-file-with-cancelation-record.dat. Error: The MSFAA file has an invalid transaction code on trailer",
],
},
]);
// Assert that the file was not deleted from SFTP.
expect(sftpClientMock.delete).not.toHaveBeenCalled();
});
},
);
5 changes: 5 additions & 0 deletions ...tion/_tests_/msfaa-receive-files/msfaa-part-time-receive-file-with-cancelation-record.dat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
100222 MSFAA RECEIVED 202111031617000001
2000000001000111111111R2021112020211121PT
2000000001001222222222C PT ON20211124
2000000001002333333333R2021112220211123PT
999MSFAA RECEIVED 000000003000000666666666
4 changes: 4 additions & 0 deletions ...n/_tests_/msfaa-receive-files/msfaa-part-time-receive-file-with-invalid-records-count.dat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
100222 MSFAA RECEIVED 202111031617000001
2000000001000111111111R2021112020211121PT
2000000001002333333333R2021112220211123PT
999MSFAA RECEIVED 000000001000000666666666
4 changes: 4 additions & 0 deletions .../_tests_/msfaa-receive-files/msfaa-part-time-receive-file-with-invalid-sin-hash-total.dat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
100222 MSFAA RECEIVED 202111031617000001
2000000001000111111111R2021112020211121PT
2000000001002333333333R2021112220211123PT
999MSFAA RECEIVED 000000002000000666666666
2 changes: 1 addition & 1 deletion ...c-integration/msfaa-integration/msfaa-part-time-process-response-integration.scheduler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ export class PartTimeMSFAAProcessResponseIntegrationScheduler extends BaseSchedu
* @returns Summary with what was processed and the list of all errors, if any.
*/
@Process()
async processMSFAA(job: Job<void>): Promise<ProcessResponseQueue[]> {
async processMSFAAResponses(job: Job<void>): Promise<ProcessResponseQueue[]> {
const summary = new QueueProcessSummary({
appLogger: this.logger,
jobLogger: job,
Expand Down
15 changes: 11 additions & 4 deletions .../integrations/src/esdc-integration/msfaa-integration/msfaa-response.processing.service.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@ import { MSFAANumberService } from "@sims/integrations/services";
import { OfferingIntensity } from "@sims/sims-db";
import { LoggerService, InjectLogger } from "@sims/utilities/logger";
import { ProcessSFTPResponseResult } from "../models/esdc-integration.model";
import { MSFAASFTPResponseFile } from "./models/msfaa-integration.model";
import {
MSFAASFTPResponseFile,
ReceivedStatusCode,
} from "./models/msfaa-integration.model";
import { MSFAAResponseCancelledRecord } from "./msfaa-files/msfaa-response-cancelled-record";
import { MSFAAResponseReceivedRecord } from "./msfaa-files/msfaa-response-received-record";
import { MSFAAIntegrationService } from "./msfaa.integration.service";
Expand Down Expand Up @@ -55,15 +58,19 @@ export class MSFAAResponseProcessingService {
return result;
}

result.processSummary.push("File contains:");
result.processSummary.push(
`File contains ${responseFile.receivedRecords.length} received records and ${responseFile.cancelledRecords.length} cancelled records.`,
`Confirmed MSFAA records (type ${ReceivedStatusCode.Received}): ${responseFile.receivedRecords.length}.`,
);
result.processSummary.push(
`Cancelled MSFAA records (type ${ReceivedStatusCode.Cancelled}): ${responseFile.cancelledRecords.length}.`,
);

for (const receivedRecord of responseFile.receivedRecords) {
try {
await this.processReceivedRecord(receivedRecord);
result.processSummary.push(
`Status record from line ${receivedRecord.lineNumber}.`,
`Record from line ${receivedRecord.lineNumber}, updated as confirmed.`,
);
} catch (error) {
// Log the error but allow the process to continue.
Expand All @@ -76,7 +83,7 @@ export class MSFAAResponseProcessingService {
try {
await this.processCancelledRecord(cancelledRecord);
result.processSummary.push(
`Status cancelled record from line ${cancelledRecord.lineNumber}.`,
`Record from line ${cancelledRecord.lineNumber}, updated as canceled.`,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cancelled ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

);
} catch (error) {
// Log the error but allow the process to continue.
Expand Down
Loading