|
| 1 | +##### LargeFileUpload Task Design Changes |
| 2 | + |
| 3 | +This document proposes high-level design modifications to the `LargeFileUploadTask` implementation. Reasons for change - |
| 4 | + |
| 5 | +- Enhancement - Support Node.js Stream upload. Issue [#320](https://github.com/microsoftgraph/msgraph-sdk-javascript/issues/320). |
| 6 | +- Bug Fix - Support large file uploads to Outlook API. Issue [#359](https://github.com/microsoftgraph/msgraph-sdk-javascript/issues/359). |
| 7 | +- Enhancement- Support upload progress handler callback. Issue [#305](https://github.com/microsoftgraph/msgraph-sdk-javascript/issues/305). |
| 8 | + |
| 9 | +Outline of the current implementation - |
| 10 | + |
| 11 | +```TypeScript |
| 12 | +interface LargeFileUploadTaskOptions { |
| 13 | + rangeSize?: number; |
| 14 | +} |
| 15 | + |
| 16 | +interface FileObject { |
| 17 | + content: ArrayBuffer | File; |
| 18 | + name: string; |
| 19 | + size: number; |
| 20 | +} |
| 21 | + |
| 22 | +// Create a LargeFileUploadTask object with the file object |
| 23 | +constructor(client: Client, file: FileObject, uploadSession: LargeFileUploadSession, options: LargeFileUploadTaskOptions = {}) |
| 24 | + |
| 25 | +// Call the LargeFileUploadTask.upload() function to upload large file to the API in ranges. |
| 26 | +upload() { |
| 27 | + while (true) { |
| 28 | + const fileSlice = this.sliceFile(nextRange); |
| 29 | + const response = await this.uploadSlice(fileSlice, nextRange, this.file.size); |
| 30 | + // Upon completion of upload process incase of onedrive, driveItem is returned, which contains id |
| 31 | + if (response.id !== undefined) { |
| 32 | + return response; |
| 33 | + } else { |
| 34 | + this.updateTaskStatus(response); |
| 35 | + } |
| 36 | + } |
| 37 | +} |
| 38 | + |
| 39 | +// Function to slice the file as per the "Next Expected Ranges". |
| 40 | +sliceFile(range: Range): ArrayBuffer | Blob { |
| 41 | + const blob = this.file.content.slice(range.minValue, range.maxValue + 1); |
| 42 | + return blob; |
| 43 | +} |
| 44 | +``` |
| 45 | + |
| 46 | +###### 1. Support Node.js Stream upload |
| 47 | + |
| 48 | +- The requirement is to allow a `FileObject` with content of type [Readable Stream](https://nodejs.org/api/stream.html#stream_class_stream_readable) |
| 49 | +- The `file.slice()` function is not applicable to `Readable Stream` and logic for splitting and handling `Stream` varies. |
| 50 | +- Note - Chunk uploading is not supported by the Graph API and the stream will be split into multiple ranges and each range will uploaded sequentially. |
| 51 | + |
| 52 | +- Proposed changes in the current design - |
| 53 | + - Move `sliceFlice()` in the FileObject Interface |
| 54 | + ```TypeScript |
| 55 | + interface FileObject{ |
| 56 | + sliceFile(range: Range): ArrayBuffer | Blob |
| 57 | + } |
| 58 | + ``` |
| 59 | + - Introduce new classes implementing the `FileObject` interface. Example - `StreamLargeFile.ts` will contain the `sliceFile` function and logic to handle stream classes. |
| 60 | + - Change in the upload function will look like |
| 61 | + ``` |
| 62 | + upload() { |
| 63 | + // current : const fileSlice = this.sliceFile(nextRange); |
| 64 | + // proposed change is as follows - |
| 65 | + const fileSlice = this.fileObject.sliceFile(Range); |
| 66 | + } |
| 67 | + ``` |
| 68 | + - This change is based on idea of Dependency Inversion principle. The goal is to depend on abstractions so that we can easily add or modify support for different file formats in the future or allow customized `FileObject` implementations. |
| 69 | +- For browser support, take a dependency on [stream-browserify](https://www.npmjs.com/package/stream-browserify) - the stream module from node core, for browsers and make necessary updates to the rollup or bundling process. |
| 70 | + |
| 71 | +###### 2. Support large file uploads to Outlook API |
| 72 | + |
| 73 | +- Bug in the current implemenation - `response.id !== undefined` this condition to mark the completion of an upload does not work for Outlook API since the final response does not contain a response body. |
| 74 | +- An upload task should be marked as completed if the response status is a 201. [SDK-design document](https://github.com/microsoftgraph/msgraph-sdk-design/blob/master/tasks/FileUploadTask.md). |
| 75 | +- The LargeFileUploadTask should allow uploads to OneDrive API, Outlook API and PrintDocument API. |
| 76 | +- Proposed changes- |
| 77 | + - Add class `UploadResult` containing `location` and `responseBody` properties. |
| 78 | + - `location` provides access to the `location` field in the response headers. |
| 79 | + - `responseBody` provides access to the Graph API response body. |
| 80 | + - The `upload` task should return the `UploadResult` object on successful completion of task. |
| 81 | + |
| 82 | +###### 3. Support upload progress handler callback |
| 83 | +- Proposed changes - |
| 84 | + - Add interface -> `interface UploadEventHandler{ |
| 85 | + extraCallbackParam?: unknown; |
| 86 | + progress(range: Range, extraCallbackParam?: unknown):void |
| 87 | + }` |
| 88 | + - Add uploadEventHandlers option to -> |
| 89 | + ``` |
| 90 | + interface LargeFileUploadTaskOptions { |
| 91 | + rangeSize?: number; |
| 92 | + uploadEventHandlers?: UploadEventHandler; |
| 93 | + } |
| 94 | + ``` |
| 95 | + - In the `upload` function call the `uploadEventHandlers.progress()` function if defined. |
0 commit comments