-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
#3254 - CAS AP Invoice - Invoices and Batches - Scheduler #4297
#3254 - CAS AP Invoice - Invoices and Batches - Scheduler #4297
Conversation
@@ -20,6 +20,7 @@ RUN npm ci | |||
# Copying sources. | |||
COPY ./apps/queue-consumers ./apps/queue-consumers | |||
COPY ./apps/db-migrations ./apps/db-migrations | |||
COPY ./apps/test-db-seeding ./apps/test-db-seeding |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
...nsumers/src/processors/schedulers/cas-integration/cas-invoices-batches-creation.scheduler.ts
Show resolved
Hide resolved
...ges/backend/apps/queue-consumers/src/services/cas-invoice-batch/cas-invoice-batch.service.ts
Outdated
Show resolved
Hide resolved
.where("disbursementReceiptValue.grantType = :grantType", { | ||
grantType: BC_TOTAL_GRANT_AWARD_CODE, | ||
}) | ||
.andWhere("disbursementReceiptValue.grantAmount > 0") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If BCSG was not available I would be more open to using the total_disbursed_grant_amount
, since we have BCSG it does seem more accurate, IMO.
BCSG represents the exact SUM of provincial grants generated by SIMS and the receipt sent back.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that we can avoid joining the disbursement_receipt_values if we can use total_disbursed_grant_amount
here. But not a blocker. Let me know if I am missing something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do understand it, but to me was clear that the check must happen using the BCSG and that would be my recommendation also. I will ask the business if they can ensure the goal will be achieved in the same way I will change.
.andWhere("disbursementValue.valueType = :valueType", { | ||
valueType: DisbursementValueType.BCGrant, | ||
}) | ||
.andWhere("disbursementValue.effectiveAmount > 0") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
...ges/backend/apps/queue-consumers/src/services/cas-invoice-batch/cas-invoice-batch.service.ts
Show resolved
Hide resolved
...ges/backend/apps/queue-consumers/src/services/cas-invoice-batch/cas-invoice-batch.service.ts
Outdated
Show resolved
Hide resolved
...ges/backend/apps/queue-consumers/src/services/cas-invoice-batch/cas-invoice-batch.service.ts
Outdated
Show resolved
Hide resolved
...ges/backend/apps/queue-consumers/src/services/cas-invoice-batch/cas-invoice-batch.service.ts
Outdated
Show resolved
Hide resolved
...ges/backend/apps/queue-consumers/src/services/cas-invoice-batch/cas-invoice-batch.service.ts
Show resolved
Hide resolved
const invoiceDetails: CASInvoiceDetail[] = []; | ||
for (const disbursedAward of disbursedAwards) { | ||
// Find the distribution accounts for the award. | ||
const accounts = distributionAccounts.filter( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will it not be useful to throw error if the distribution account size is 0?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It may, but I will raise the question and leave it to a second PR.
It may be useful if it is 0 records, or if there is only one, or if it is not a pair of debit and credit operations or they may want the ability to not have any and avoid including it in the invoice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will be taken care on an upcoming PR as per business agreement. Just some basic validation.
sequenceName: CAS_INVOICE_SEQUENCE_NAME, | ||
}, | ||
}); | ||
expect(currentInvoiceSequenceNumber.sequenceNumber).toEqual("2"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
}); | ||
}); | ||
|
||
it("Should create a new CAS invoice batch for a full-time receipt when there is a e-Cert receipt with no invoice associated.", async () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
E2E looks great. Unless there are more E2E tests going to come here in upcoming PRs, I would ask to have an E2E with no pending invoices to process.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There will be more E2Es. The next PR will be for UI, but UI and schedulers must receive more E2Es.
operationCode: "CR" | "DR"; | ||
} | ||
|
||
export const CAS_DISTRIBUTION_ACCOUNTS_INITIAL_DATE: CASDistributionAccountBaseData[] = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is great idea to have it data seeded.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea is to share them with the API E2E tests also in upcoming PRs in this sprint.
sequenceRecord.sequenceNumber = nextSequenceNumber.toString(); | ||
const result = await process(nextSequenceNumber, entityManager); | ||
if (typeof result === "number") { | ||
// If the external process returned a number, it will be used as the new sequence number. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
|
await this.sequenceControlService.consumeNextSequenceWithExistingEntityManager( | ||
"CAS_INVOICE", | ||
entityManager, | ||
async (nextSequenceNumber: number) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can be explicit with the return type here.
async (nextSequenceNumber: number): Promise<number>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for doing the changes @andrewsignori-aot. Looks good. One last minor comment.
As discussed, if there is any change based on response from business, we can have that in upcoming PR.
@@ -33,6 +38,7 @@ import { CreateRestrictions } from "./db-seeding/restriction"; | |||
CreateAESTUsers, | |||
CreateStudentUsers, | |||
CreateRestrictions, | |||
CreateCASDistributionAccounts, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
@@ -31,7 +31,7 @@ | |||
"test:e2e:workers:local": "cross-env ENVIRONMENT=test jest --collect-coverage --verbose --config ./apps/workers/test/jest-e2e.json --forceExit", | |||
"test:e2e:workflow": "npm run deploy:camunda:definitions && cross-env ENVIRONMENT=test jest --verbose --config ./workflow/test/jest-e2e.json --forceExit", | |||
"test:e2e:workflow:local": "cross-env ENVIRONMENT=test TZ=UTC jest --verbose --config ./workflow/test/jest-e2e.json --forceExit", | |||
"test:e2e:queue-consumers": "npm run migration:run && cross-env ENVIRONMENT=test jest --collect-coverage --verbose --config ./apps/queue-consumers/test/jest-e2e.json --runInBand --forceExit", | |||
"test:e2e:queue-consumers": "npm run db:seed:test filter=CreateCASDistributionAccounts && cross-env ENVIRONMENT=test jest --collect-coverage --verbose --config ./apps/queue-consumers/test/jest-e2e.json --runInBand --forceExit", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, nice work @andrewsignori-aot
Created new E2E tests for the new endpoints and extended scheduler E2E tests previously added (only one). ## API Endpoints (CASInvoiceBatchAESTController) - getCASInvoiceBatchReport - Should generate an invoice batch report with part-time and full-time invoices when the batch exists. - Should throw a HttpStatus Not Found (404) when the requested invoice batch does not exist. - Should throw a HttpStatus Forbidden (403) error when an unauthorized Ministry user tries to get the invoice batch report. - getInvoiceBatches - Should be able to get invoice batches for the first page in a paginated result with a limit of two per page in descending order when there are three invoice batches available. - Should be able to get only pending invoice batches when the filter for status is applied. - Should throw a HttpStatus Bad Request (400) error when the filter for status is invalid. - Should throw a HttpStatus Bad Request (400) error when the sortField is invalid. - Should throw a HttpStatus Forbidden (403) error when an unauthorized Ministry user tries to get the invoice batches. ## Scheduler - Should create a new CAS invoice and avoid making a second invoice when a receipt already has an invoice associated with it. - Should interrupt the process when an invoice is trying to be generated but there are no distribution accounts available to create the invoice details ([addressing previous PR comment](#4297 (comment))). - Should finalize the process nicely when there is no pending receipt to process ([addressing previous PR comment](#4297 (comment))).
console
.sequence-control
service was changed to allow outside control of the sequence increment.Pending Receipts Query
Saving Operations
The method to save the invoices is using a single transaction and chunk inserts. The 150-chunk size was determined by local tests using 50 to 500-chunk sizes. The tests were realized with 1000 invoice records and each one had 4 invoice details where it took around 1 second to complete the DB persistence operation. In terms of SQL parameters, Postgres can accept a maximum of 65K parameters in a single command.
The save operation in chunks was not logging any additional select and was grouping the insert operations as below for invoices and similar for invoice details.
Migration revert