Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#3738 - SIMS to SFAS Part 2 #3829

Merged
merged 33 commits into from
Oct 28, 2024
Merged

Conversation

dheepak-aot
Copy link
Collaborator

@dheepak-aot dheepak-aot commented Oct 24, 2024

SIMS TO SFAS - INTEGRATION IMPLEMENTATION FOR STUDENT DATA RECORDS

ENV

  • Added a new env variable SFAS_SEND_FOLDER. Declared it as github env, not as secret.

PROCESS

  • File generated with following name structure SIMS-TO-SFAS-YYYYMMDD-HHMMSS.TXT
    image

  • Student data is retrieved to build the file for all students who has one or more of the following updates since last run date.

    • Student or User data
    • Sin validation data
    • Cas supplier data
    • Overawards data
  • Students with at least one submitted application are considered to extract the data. As submitted applications can be cancelled and also the Draft applications can be cancelled, application status NOT Draft could be tricky. So used the condition of application with current assessment NOT NULL to identify a submitted application.

  • When one or more students with updates are present, then a file is produced and sent to SFAS SFTP location.
    Summary:
    image

Process Logs:
image

  • When no students with updates are present, then no file is produced.

Summary:
image

Process Logs:

image

TECHNICAL CONTEXT

  • STEPS involved in creation of the bridge data file
STEP 1 [Implemented]: Get the reference date of last bridge file sent. If there is no bridge file sent, it is set to a safe date where all the data in SIMS can be initially loaded in the file.

STEP 2 [Implemented]: Get all the student ids of students who has one or more student data related change since last bridge file date.

STEP 3 [TODO]: Get all the student ids and application details of students who has one or more application data related change since last bridge file date.

STEP 4 [TODO]: Get all the student ids and restriction details of students who has one or more restriction data related change since last bridge file date.

STEP 5 [Partially Implemented]: Consolidate all the studentIds produced in STEPS 2,3 and 4. 

STEP 6 [Partially Implemented]: Get student details of students who has one or more student data related change using consolidated studentIds.

STEP 7 [Implemented]: Build header, footer and student records for file lines.

STEP 8 [TODO]: Build application records and restriction records for file lines.

STEP 9[Implemented]: Upload the file to SFTP and create bridge file log.

  • For future implementations and areas, search for TODO: SIMS to SFAS in the code.

@dheepak-aot dheepak-aot marked this pull request as ready for review October 24, 2024 23:11
@guru-aot guru-aot self-requested a review October 28, 2024 16:08
Copy link
Collaborator

@andrewsignori-aot andrewsignori-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work and great start to the SFAS bridge. Looks good 👍

// Check if the student data was updated in the given period.
.andWhere(
new Brackets((qb) => {
qb.where(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

[
"Process finalized with success.",
`Student records sent: ${studentRecordsSent}`,
`Uploaded file name: ${uploadedFileName}`,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add a period at the end??

.appendHeader(bridgeFileDate)
.appendStudentFileRecords(studentRecords)
.appendFooter().fileLines;
// TODO: SIMS to SFAS - Append applications, restrictions and footer.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

// Append the students with student and student related data updates.
// TODO: SIMS to SFAS - Append the student ids of students with updates in application and restriction related data.
// When application and restriction updates are retrieved, the respective
// student ids of applications and restrictions should be appended.;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the semi colon at the end

Copy link

Copy link

Backend Unit Tests Coverage Report

Totals Coverage
Statements: 22.35% ( 3692 / 16516 )
Methods: 10.33% ( 212 / 2053 )
Lines: 25.67% ( 3202 / 12475 )
Branches: 13.98% ( 278 / 1988 )

Copy link

E2E Workflow Workers Coverage Report

Totals Coverage
Statements: 65.43% ( 583 / 891 )
Methods: 59.26% ( 64 / 108 )
Lines: 68.54% ( 464 / 677 )
Branches: 51.89% ( 55 / 106 )

Copy link

E2E Queue Consumers Coverage Report

Totals Coverage
Statements: 83.33% ( 1180 / 1416 )
Methods: 84.29% ( 118 / 140 )
Lines: 84.27% ( 1002 / 1189 )
Branches: 68.97% ( 60 / 87 )

Copy link
Collaborator

@guru-aot guru-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great work @dheepak-aot

Copy link

E2E SIMS API Coverage Report

Totals Coverage
Statements: 66.01% ( 5571 / 8439 )
Methods: 63.3% ( 683 / 1079 )
Lines: 70.12% ( 4401 / 6276 )
Branches: 44.93% ( 487 / 1084 )

@dheepak-aot dheepak-aot added this pull request to the merge queue Oct 28, 2024
Merged via the queue into main with commit 7eedbb5 Oct 28, 2024
20 checks passed
@dheepak-aot dheepak-aot mentioned this pull request Oct 29, 2024
20 tasks
@dheepak-aot dheepak-aot deleted the feature/#3738-sims-to-sfas-part-2 branch November 4, 2024 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants