Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1834 - Service task identifier for subprocesses #1842

Merged

Conversation

andrewsignori-aot
Copy link
Collaborator

@andrewsignori-aot andrewsignori-aot commented Mar 28, 2023

Test cases enabled

To provide complete support for mocking the workers, and allowing the workflow to be tested including the subprocesses, some scenarios from parents were used. As part of this PR the below scenarios were enabled:

  • √ Should check for both parents incomes when the student is dependant and parents have SIN.
  • √ Should check for parent 1 income when the student is dependant and informed that he has only one parent with SIN.
  • √ Should skip parent income verification when the student is dependant and informed that parents do not have SIN.

Single Mocked Worker

Created a single mocked worker method to return the job completed object and also publish messages needed to unblock the workflow.
While creating the mocked result for a worker, it is possible to specify the object to be returned to Camunda and also configure 1 to n messages that must be published to unblock the workflow. As shown below, the cra-integration-income-verification needs to be completed using 3 mocked objects created using 2 helpers.

  1. The Create income request must have some data returned;
  2. Once the Create income request is completed, the workflow must receive a message, also mocked and shown on the below message.
  3. During step number 3, the workflow must receive a mocked response with the incomeVerificationCompleted as true to proceed.

image

Validation/Asserts

image

Examples of variables available on the workflow to perform the validations related to the flow path expectToPassThroughServiceTasks and expectNotToPassThroughServiceTasks.

{
   "program-info-not-required-task": "program-info-not-required-task",
   "associate-workflow-instance-task": "associate-workflow-instance-task",
    "update-noa-status-to-required-task": "update-noa-status-to-required-task",
    "verify-application-exceptions-task": "verify-application-exceptions-task",
    "parent1IncomeVerificationSubprocess": "parent1IncomeVerificationSubprocess",
    "parent2IncomeVerificationSubprocess": "parent2IncomeVerificationSubprocess",
    "studentIncomeVerificationSubprocess": "studentIncomeVerificationSubprocess",
    "retrieveSupportingInfoParent1Subprocess": "retrieveSupportingInfoParent1Subprocess",
    "retrieveSupportingInfoParent2Subprocess": "retrieveSupportingInfoParent2Subprocess",
}

The value that indicates if the service task or subprocess was changed from true to the string above makes it easier to identify which validation failed, as shown below.

image

Camunda bpmn configurations for subprocess identification

image

Sample mocked start event data

Scenario:

  • Create 2 supporting users;
  • Create 2 parent-provided information;
  • Execute 3 income verifications (student and 2 parents).
{
    "bpmnProcessId": "assessment-gateway",
    "variables": {
        "assessmentId": 2000,
        "load_assessment_data_task_result": {
            "data_removed_to_make_it_short": "Original assessment",
        },
        "verify_application_exceptions_task_result": {
            "applicationExceptionStatus": "Approved"
        },
        "create_supporting_users_for_parents_task_result": {
            "createdSupportingUsersIds": [
                2000,
                2001
            ]
        },
        "create_supporting_users_for_parents_task_messageResult": [
            {
                "name": "supporting-user-info-received",
                "correlationKey": "2000",
                "variables": {},
                "timeToLive": {
                    "type": "SECONDS",
                    "value": 5,
                    "valueType": "TYPED_DURATION",
                    "unit": "s"
                }
            },
            {
                "name": "supporting-user-info-received",
                "correlationKey": "2001",
                "variables": {},
                "timeToLive": {
                    "type": "SECONDS",
                    "value": 5,
                    "valueType": "TYPED_DURATION",
                    "unit": "s"
                }
            }
        ],
        "retrieveSupportingInfoParent1Subprocess_check_supporting_user_response_task_result": {
            "totalIncome": 1
        },
        "retrieveSupportingInfoParent2Subprocess_check_supporting_user_response_task_result": {
            "totalIncome": 1
        },
        "studentIncomeVerificationSubprocess_create_income_request_task_result": {
            "incomeVerificationCompleted": true,
            "incomeVerificationId": 2000
        },
        "studentIncomeVerificationSubprocess_create_income_request_task_messageResult": [
            {
                "name": "income-verified",
                "correlationKey": "2000",
                "variables": {},
                "timeToLive": {
                    "type": "SECONDS",
                    "value": 5,
                    "valueType": "TYPED_DURATION",
                    "unit": "s"
                }
            }
        ],
        "parent1IncomeVerificationSubprocess_create_income_request_task_result": {
            "incomeVerificationCompleted": true,
            "incomeVerificationId": 2001
        },
        "parent1IncomeVerificationSubprocess_create_income_request_task_messageResult": [
            {
                "name": "income-verified",
                "correlationKey": "2001",
                "variables": {},
                "timeToLive": {
                    "type": "SECONDS",
                    "value": 5,
                    "valueType": "TYPED_DURATION",
                    "unit": "s"
                }
            }
        ],
        "parent2IncomeVerificationSubprocess_create_income_request_task_result": {
            "incomeVerificationCompleted": true,
            "incomeVerificationId": 2002
        },
        "parent2IncomeVerificationSubprocess_create_income_request_task_messageResult": [
            {
                "name": "income-verified",
                "correlationKey": "2002",
                "variables": {},
                "timeToLive": {
                    "type": "SECONDS",
                    "value": 5,
                    "valueType": "TYPED_DURATION",
                    "unit": "s"
                }
            }
        ],
        "studentIncomeVerificationSubprocess_check_income_request_task_result": {
            "incomeVerificationCompleted": true
        },
        "parent1IncomeVerificationSubprocess_check_income_request_task_result": {
            "incomeVerificationCompleted": true
        },
        "parent2IncomeVerificationSubprocess_check_income_request_task_result": {
            "incomeVerificationCompleted": true
        }
    },
    "requestTimeout": 90000
}

Variable names were adapted trying to follow Camunda recommendations for variables names: https://docs.camunda.io/docs/components/concepts/variables/#variable-names

@andrewsignori-aot andrewsignori-aot added E2E/Unit tests Camunda Worflow Involves camunda workflow changes labels Apr 1, 2023
@andrewsignori-aot andrewsignori-aot self-assigned this Apr 1, 2023
@andrewsignori-aot andrewsignori-aot changed the title #1834 - Service task identifier sub process #1834 - Service task identifier for subprocesses Apr 1, 2023
@guru-aot
Copy link
Collaborator

guru-aot commented Apr 3, 2023

The e2e test cases for assessment gatewat for program year 2021-2022 is removed. Is that because those years are no longer used and its just a duplication?

Comment on lines +293 to +298
WorkflowSubprocesses.PartnerIncomeVerification,
WorkflowSubprocesses.RetrieveSupportingInfoParent1,
WorkflowSubprocesses.Parent1IncomeVerification,
WorkflowSubprocesses.RetrieveSupportingInfoParent2,
WorkflowSubprocesses.Parent2IncomeVerification,
);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

* @returns mock for the 'Load consolidated data' completed subprocess.
*/
export function createLoadAssessmentConsolidatedDataMock(options: {
assessmentConsolidatedData: AssessmentConsolidatedData;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment missing

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

const assessmentConsolidatedData: AssessmentConsolidatedData = {
assessmentTriggerType: AssessmentTriggerType.OriginalAssessment,
...createFakeConsolidatedFulltimeData(PROGRAM_YEAR),
...createParentsData({ numberOfParents: 2 }),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this way works, but the supporting user parent data must be returned by the following worker and not the load-assessment-consolidated data right?

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that there is confusion here. The service task above has specific mocked data to return its data.
The assumption that parent data is returned by the workflow in the picture is not correct.
All the data (I mean, all the data) to process an assessment (and reassessments) is returned in the consolidated data.
Keep in mind that the reassessment does not go down this path and if this was the workflow responsible for loading the parent's data the reassessment would never have supporting users' data.
Please let me know If it is clear or if I am missing something.

Copy link
Contributor

@ann-aot ann-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @andrewsignori-aot 👍 . Added some minor comments. I believe I got the idea, let me know if you planning to do a walkthrough, I would also like to join, to cover up in case I miss something.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Apr 3, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@github-actions
Copy link

github-actions bot commented Apr 3, 2023

Backend Unit Tests Coverage Report

Totals Coverage
Statements: 18.01% ( 1957 / 10865 )
Methods: 8.14% ( 115 / 1413 )
Lines: 20.71% ( 1713 / 8271 )
Branches: 10.92% ( 129 / 1181 )

@github-actions
Copy link

github-actions bot commented Apr 3, 2023

E2E Workflow Workers Coverage Report

Totals Coverage
Statements: 32.41% ( 176 / 543 )
Methods: 21.25% ( 17 / 80 )
Lines: 39.25% ( 157 / 400 )
Branches: 3.17% ( 2 / 63 )

@github-actions
Copy link

github-actions bot commented Apr 3, 2023

E2E Queue Consumers Coverage Report

Totals Coverage
Statements: 56.72% ( 308 / 543 )
Methods: 46.38% ( 32 / 69 )
Lines: 59.48% ( 276 / 464 )
Branches: 0% ( 0 / 10 )

@github-actions
Copy link

github-actions bot commented Apr 3, 2023

E2E SIMS API Coverage Report

Totals Coverage
Statements: 38.07% ( 2565 / 6738 )
Methods: 29.84% ( 262 / 878 )
Lines: 43.65% ( 2194 / 5026 )
Branches: 13.07% ( 109 / 834 )

serviceTasks.forEach((serviceTask) => {
expect(workflowResultVariables[serviceTask]).toBeUndefined;
expectedTasks.forEach((expectedTask) => {
expect(workflowResultVariables[expectedTask]).toBeUndefined();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

* @param assessmentConsolidatedData assessment consolidated data.
* @returns mock for the 'Load consolidated data' completed subprocess.
*/
export function createLoadAssessmentConsolidatedDataMock(options: {
Copy link
Collaborator

@dheepak-aot dheepak-aot Apr 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is options not optional ?. May be assessmentConsolidatedData not one of the options?

@dheepak-aot
Copy link
Collaborator

dheepak-aot commented Apr 4, 2023

Great work and thanks for the walkthrough.

In github actions I saw these logs which I haven't seen before.

image

Still have doubts on about #1842 (comment) and we can talk to get my understanding better.

*/
export function createCheckSupportingUserResponseTaskMock(options: {
totalIncome: number;
subprocesses?: WorkflowSubprocesses;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My be we could name it subprocess as we not take an array.

* Camunda variable name like service_task_id.
*/
function getNormalizedServiceTaskId(serviceTaskId: string) {
return serviceTaskId.replace(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got the idea of camunda variables not allowed with -. Why cant we change the service task id to have underscore _instead of - as convention so that we can avoid this replace, and also when we create any task in modeler the default task id which is generated is with _ and not -.

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would assume that if we will be changing the way that we are using the IDs we would be changing everywhere, not limited to only tasks, and also renaming the subprocesses calls.
We can do it but I do not believe that it would be the scope of this PR.


/**
* Creates the mock for the 'Load consolidated data' completed subprocess.
* @param assessmentConsolidatedData assessment consolidated data.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@param options

Copy link
Collaborator

@andrepestana-aot andrepestana-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! I left some comments

*/
export function createMockedWorkerResult(
serviceTaskId: WorkflowServiceTasks,
options: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't options be optional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this options is not optional because ideally one of the three parameters should be provided.

Comment on lines +158 to +164
if (options?.subprocesses?.length) {
fullServiceTaskId = [...options.subprocesses, fullServiceTaskId].join(
MOCKS_SEPARATOR,
);
}
const mockedWorkerResult: Record<string, unknown> = {};
if (options.jobCompleteMock) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either options is optional and we do the check with "?" or we don't need to check for null.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a comment for line 164?
For line 164 options will not be null but options.jobCompleteMock can be null.
The comment would make sense for line 158 where we can have it as if (options.subprocesses?.length). Make sense?

Comment on lines +22 to +24
totalIncome: options.totalIncome,
},
subprocesses: [options?.subprocesses],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

@guru-aot guru-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work @andrewsignori-aot

Copy link
Contributor

@ann-aot ann-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @andrewsignori-aot . Thanks for doing the changes and working on organizing the structure 👍

Copy link
Collaborator

@dheepak-aot dheepak-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the walk through and nice discussions. LGTM. 👍

Copy link
Collaborator

@andrepestana-aot andrepestana-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation about how it works.

@andrewsignori-aot andrewsignori-aot merged commit 2145c5b into main Apr 5, 2023
@andrewsignori-aot andrewsignori-aot temporarily deployed to DEV April 5, 2023 17:40 — with GitHub Actions Inactive
@andrewsignori-aot andrewsignori-aot temporarily deployed to DEV April 5, 2023 17:40 — with GitHub Actions Inactive
@andrewsignori-aot andrewsignori-aot deleted the feature/#1834-service-task-identifier-sub-process branch April 5, 2023 17:40
@andrewsignori-aot andrewsignori-aot temporarily deployed to DEV April 5, 2023 17:50 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Camunda Worflow Involves camunda workflow changes E2E/Unit tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants