Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#2111-UI format validation bulk withdrawal upload #2209

Merged
merged 40 commits into from
Aug 25, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7653b6c
INitial commit
guru-aot Aug 14, 2023
6e9d841
updated
guru-aot Aug 16, 2023
d50375e
updated
guru-aot Aug 18, 2023
d63bd89
updated
guru-aot Aug 18, 2023
732096c
Merge branch 'main' into #2111-UI-Format-Validation-Bulk-Withdrawal-U…
guru-aot Aug 18, 2023
5b5d37e
updated
guru-aot Aug 18, 2023
12048ad
updated
guru-aot Aug 18, 2023
21c4eaa
updated
guru-aot Aug 18, 2023
836db70
updated
guru-aot Aug 18, 2023
8541288
updated
guru-aot Aug 21, 2023
177f80f
updated
guru-aot Aug 21, 2023
543a21e
updated
guru-aot Aug 21, 2023
8aba933
updated
guru-aot Aug 21, 2023
15828df
updated
guru-aot Aug 22, 2023
f68ff61
updated
guru-aot Aug 22, 2023
f5e5207
updated
guru-aot Aug 23, 2023
df2467f
updated
guru-aot Aug 23, 2023
2b14c20
updated
guru-aot Aug 23, 2023
5640b08
updated
guru-aot Aug 23, 2023
85f4996
updated
guru-aot Aug 23, 2023
3251f87
updated
guru-aot Aug 23, 2023
de8207c
updated
guru-aot Aug 23, 2023
c0cbdf7
updated
guru-aot Aug 23, 2023
6569267
updated
guru-aot Aug 23, 2023
26d7152
updated
guru-aot Aug 23, 2023
7fe36e5
updated
guru-aot Aug 23, 2023
b230d38
updated
guru-aot Aug 23, 2023
2dc8b4e
updated
guru-aot Aug 24, 2023
85c236e
updated
guru-aot Aug 24, 2023
9754717
updated
guru-aot Aug 24, 2023
b0caa5c
updated
guru-aot Aug 24, 2023
a921b71
updated
guru-aot Aug 24, 2023
0f6c531
updated
guru-aot Aug 24, 2023
473c319
updated
guru-aot Aug 25, 2023
bf400a4
updated
guru-aot Aug 25, 2023
3eca7cd
updated
guru-aot Aug 25, 2023
647edee
updated
guru-aot Aug 25, 2023
632971c
updated
guru-aot Aug 25, 2023
7901b0a
updated
guru-aot Aug 25, 2023
9b53153
Merge branch 'main' into #2111-UI-Format-Validation-Bulk-Withdrawal-U…
guru-aot Aug 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import {
ApplicationExceptionService,
StudentAppealRequestsService,
ApplicationOfferingChangeRequestService,
ApplicationWithdrawalImportTextService,
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
} from "./services";
import {
ApplicationControllerService,
Expand Down Expand Up @@ -162,6 +163,7 @@ import {
ApplicationExceptionControllerService,
StudentAppealControllerService,
ApplicationOfferingChangeRequestService,
ApplicationWithdrawalImportTextService,
],
})
export class AppInstitutionsModule {}
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,17 @@ export const OFFERING_PROGRAM_YEAR_MISMATCH = "OFFERING_PROGRAM_YEAR_MISMATCH";
*/
export const OFFERING_DOES_NOT_BELONG_TO_LOCATION =
"OFFERING_DOES_NOT_BELONG_TO_LOCATION";
/**
* The text content to perform the application withdrawal is not in the
* expected format and cannot be parsed.
*/
export const APPLICATION_WITHDRAWAL_TEXT_CONTENT_FORMAT_ERROR =
dheepak-aot marked this conversation as resolved.
Show resolved Hide resolved
"APPLICATION_WITHDRAWAL_TEXT_CONTENT_FORMAT_ERROR";
/**
* The text content has invalid content, either
* 1. Invalid Header record type.
* 2. Invalid Footer record type.
* 3. Number of records in the footer does not match the no of data records.
*/
export const APPLICATION_WITHDRAWAL_INVALID_TEXT_FILE_ERROR =
"APPLICATION_WITHDRAWAL_INVALID_TEXT_FILE_ERROR";
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,9 @@ import { Allow, IsEnum, IsIn, IsNotEmpty, MaxLength } from "class-validator";
import { Type } from "class-transformer";
import {
OfferingDeliveryOptions,
OfferingValidationInfos,
OfferingValidationWarnings,
WILComponentOptions,
} from "../../../services";
import { ValidationResultAPIOutDTO } from "../../models/common.dto";

export class StudyBreakAPIOutDTO {
breakStartDate: string;
Expand Down Expand Up @@ -230,14 +229,3 @@ export class OfferingBulkInsertValidationResultAPIOutDTO {
infos: ValidationResultAPIOutDTO[];
warnings: ValidationResultAPIOutDTO[];
}

/**
* Represents an error considered not critical for
* an offering and provides a user-friendly message
* and a type that uniquely identifies this warning
* or info.
*/
export class ValidationResultAPIOutDTO {
typeCode: OfferingValidationWarnings | OfferingValidationInfos;
message: string;
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
import { IsNotEmpty, IsOptional, ValidateIf } from "class-validator";
import { COUNTRY_CANADA, OTHER_COUNTRY } from "../utils/address-utils";
import {
OfferingValidationInfos,
dheepak-aot marked this conversation as resolved.
Show resolved Hide resolved
OfferingValidationWarnings,
} from "../../services";

/**
* Common DTO for Address.
Expand Down Expand Up @@ -111,3 +115,15 @@ export class FileCreateAPIOutDTO {
size: number;
mimetype: string;
}

/**
* Represents an error considered not critical for
* an offering or application withdrawal
* and provides an user-friendly message
* and a type that uniquely identifies this warning
* or info.
*/
export class ValidationResultAPIOutDTO {
typeCode: OfferingValidationWarnings | OfferingValidationInfos | string;
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
message: string;
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { IsNotEmptyObject } from "class-validator";
import { ActiveApplicationDataAPIOutDTO } from "../../../route-controllers/institution-locations/models/application.dto";
import { JSON_10KB } from "../../../constants";
import { StudentScholasticStandingChangeType } from "@sims/sims-db";
import { ValidationResultAPIOutDTO } from "../../models/common.dto";

/**
* The API will also allow other properties that are not added below.
Expand Down Expand Up @@ -35,3 +36,18 @@ export class ScholasticStandingSubmittedDetailsAPIOutDTO extends IntersectionTyp
ScholasticStandingData,
ActiveApplicationDataAPIOutDTO,
) {}

/**
* Represents the possible errors that can happen during the
* application bulk withdrawal and provides a detailed description
* for every record that has an error.
*/
export interface ApplicationBulkWithdrawalValidationResultAPIOutDTO {
recordIndex: number;
sin?: string;
dheepak-aot marked this conversation as resolved.
Show resolved Hide resolved
applicationNumber?: string;
withdrawalDate?: Date;
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
errors: string[];
infos: ValidationResultAPIOutDTO[];
warnings: ValidationResultAPIOutDTO[];
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,17 @@
import { Injectable, NotFoundException } from "@nestjs/common";
import {
BadRequestException,
Injectable,
NotFoundException,
} from "@nestjs/common";
import { StudentScholasticStandingsService } from "../../services";
import { ScholasticStandingSubmittedDetailsAPIOutDTO } from "./models/student-scholastic-standings.dto";
import {
ApplicationBulkWithdrawalValidationResultAPIOutDTO,
ScholasticStandingSubmittedDetailsAPIOutDTO,
} from "./models/student-scholastic-standings.dto";
import { transformToActiveApplicationDataAPIOutDTO } from "../institution-locations/models/application.dto";
import { ApiProcessError } from "../../types";
import { APPLICATION_WITHDRAWAL_TEXT_CONTENT_FORMAT_ERROR } from "../../constants";
import { ApplicationWithdrawalTextValidationResult } from "../../services/application-bulk-withdrawal/application-bulk-withdrawal-text.models";

/**
* Scholastic standing controller service.
Expand Down Expand Up @@ -40,4 +50,36 @@ export class ScholasticStandingControllerService {
...transformToActiveApplicationDataAPIOutDTO(application, offering),
};
}
/**
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
* Verify if all text file validations were performed with success and throw
* a BadRequestException in case of some failure.
* @param textValidations validations to be verified.
*/
assertTextValidationsAreValid(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this method written in controller service? is there an intention to share it between controllers of different client types?

Copy link
Collaborator Author

@guru-aot guru-aot Aug 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed now

textValidations: ApplicationWithdrawalTextValidationResult[],
) {
const textValidationsErrors = textValidations.filter(
(textValidation) => textValidation.errors.length,
);
if (textValidationsErrors.length) {
// At least one error was detected and the text must be fixed.
const validationResults: ApplicationBulkWithdrawalValidationResultAPIOutDTO[] =
textValidationsErrors.map((validation) => ({
recordIndex: validation.index,
sin: validation.textModel.sin,
applicationNumber: validation.textModel.applicationNumber,
withdrawalDate: validation.textModel.withdrawalDate,
errors: validation.errors,
infos: [],
warnings: [],
}));
throw new BadRequestException(
new ApiProcessError(
"One or more text data fields received are not in the correct format.",
APPLICATION_WITHDRAWAL_TEXT_CONTENT_FORMAT_ERROR,
validationResults,
),
);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,16 @@ import {
BadRequestException,
Body,
Controller,
DefaultValuePipe,
Get,
Param,
ParseBoolPipe,
ParseIntPipe,
Post,
Query,
UnprocessableEntityException,
UploadedFile,
UseInterceptors,
} from "@nestjs/common";
import {
ApiBadRequestResponse,
Expand All @@ -19,11 +24,13 @@ import { IInstitutionUserToken } from "../../auth/userToken.interface";
import {
AllowAuthorizedParty,
HasLocationAccess,
IsInstitutionAdmin,
UserToken,
} from "../../auth/decorators";
import {
APPLICATION_NOT_FOUND,
ASSESSMENT_ALREADY_IN_PROGRESS,
ApplicationWithdrawalImportTextService,
FormService,
INVALID_OPERATION_IN_THE_CURRENT_STATUS,
StudentAssessmentService,
Expand All @@ -33,13 +40,26 @@ import { ApiProcessError, ClientTypeBaseRoute } from "../../types";
import { CustomNamedError } from "@sims/utilities";
import BaseController from "../BaseController";
import { FormNames } from "../../services/form/constants";
import { APPLICATION_CHANGE_NOT_ELIGIBLE } from "../../constants";
import {
APPLICATION_CHANGE_NOT_ELIGIBLE,
APPLICATION_WITHDRAWAL_INVALID_TEXT_FILE_ERROR,
} from "../../constants";
import {
ScholasticStandingAPIInDTO,
ScholasticStandingSubmittedDetailsAPIOutDTO,
} from "./models/student-scholastic-standings.dto";
import { ScholasticStandingControllerService } from "./student-scholastic-standings.controller.service";
import { ScholasticStanding } from "../../services/student-scholastic-standings/student-scholastic-standings.model";
import { FileInterceptor } from "@nestjs/platform-express";
import {
APPLICATION_BULK_WITHDRAWAL_MAX_UPLOAD_PARTS,
APPLICATION_BULK_WITHDRAWAL_UPLOAD_MAX_FILE_SIZE,
MAX_UPLOAD_FILES,
textFileFilter,
uploadLimits,
} from "../../utilities";
import { PrimaryIdentifierAPIOutDTO } from "../models/primary.identifier.dto";
import { ApplicationWithdrawalTextModel } from "../../services/application-bulk-withdrawal/application-bulk-withdrawal-text.models";

/**
* Scholastic standing controller for institutions Client.
Expand All @@ -53,6 +73,7 @@ export class ScholasticStandingInstitutionsController extends BaseController {
private readonly studentScholasticStandingsService: StudentScholasticStandingsService,
private readonly studentAssessmentService: StudentAssessmentService,
private readonly scholasticStandingControllerService: ScholasticStandingControllerService,
private readonly applicationWithdrawalImportTextService: ApplicationWithdrawalImportTextService,
) {
super();
}
Expand Down Expand Up @@ -139,4 +160,83 @@ export class ScholasticStandingInstitutionsController extends BaseController {
userToken.authorizations.getLocationsIds(),
);
}
/**
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
* Process a text file with application to be withdrawn.
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
* @param file text file content with all information needed to withdraw application.
andrepestana-aot marked this conversation as resolved.
Show resolved Hide resolved
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
* @param validationOnly if true, will execute all validations and return the
* errors and warnings. These validations are the same executed during the
* final creation process. If not present or false, the file will be processed
* and the records will be inserted.
* @returns when successfully executed, the list of all withdrawn application ids.
* When an error happen it will return all the records (with the error) and
* also a user friendly description of the errors to be fixed.
*/
@ApiBadRequestResponse({
description:
"Error while parsing text file or " +
"one or more text data fields received are not in the correct format or " +
"there are no records to be imported.",
})
@ApiUnprocessableEntityResponse({
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
description:
"Application withdrawal has invalid data or " +
"some error happen with one or more application withdrawal and the entire process was aborted.",
})
@IsInstitutionAdmin()
@Post("application-bulk-withdrawal")
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
@UseInterceptors(
FileInterceptor("file", {
limits: uploadLimits(
MAX_UPLOAD_FILES,
APPLICATION_BULK_WITHDRAWAL_MAX_UPLOAD_PARTS,
APPLICATION_BULK_WITHDRAWAL_UPLOAD_MAX_FILE_SIZE,
),
fileFilter: textFileFilter,
}),
)
async bulkWithdrawal(
@UploadedFile() file: Express.Multer.File,
dheepak-aot marked this conversation as resolved.
Show resolved Hide resolved
@Query("validation-only", new DefaultValuePipe(false), ParseBoolPipe)
validationOnly: boolean,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my info, can we have a scenario when we process a file without doing the validations?

Copy link
Collaborator Author

@guru-aot guru-aot Aug 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think, even with business validations it will not do the existing validations.

): Promise<PrimaryIdentifierAPIOutDTO[]> {
// Read the entire file content.
const fileContent = file.buffer.toString();
// Convert the file raw content into text models.
let textModels: ApplicationWithdrawalTextModel[];
try {
textModels =
this.applicationWithdrawalImportTextService.readText(fileContent);
ann-aot marked this conversation as resolved.
Show resolved Hide resolved
} catch (error: unknown) {
let errorMessage = "Error while parsing text file.";
dheepak-aot marked this conversation as resolved.
Show resolved Hide resolved
if (
error instanceof CustomNamedError &&
error.name === APPLICATION_WITHDRAWAL_INVALID_TEXT_FILE_ERROR
) {
errorMessage = error.message;
}
throw new BadRequestException(
new ApiProcessError(
errorMessage,
APPLICATION_WITHDRAWAL_INVALID_TEXT_FILE_ERROR,
),
);
}
dheepak-aot marked this conversation as resolved.
Show resolved Hide resolved
// Validate the text models.
const textValidations =
this.applicationWithdrawalImportTextService.validateTextModels(
textModels,
);
// Assert successful validation.
this.scholasticStandingControllerService.assertTextValidationsAreValid(
textValidations,
);

ann-aot marked this conversation as resolved.
Show resolved Hide resolved
if (validationOnly) {
// If the endpoint is called only to perform the validation and no error was found
// return an empty array because no record will be created.
return [];
}

// TODO create a block to do database updates for Application withdrawal.
andrepestana-aot marked this conversation as resolved.
Show resolved Hide resolved
dheepak-aot marked this conversation as resolved.
Show resolved Hide resolved
}
}
Loading