skips lists with certain templates from being backed-up #4931

HiteshRepo · 2023-12-23T15:03:32Z

skips documentLibrary templated lists from:

restoration
details

but the lists with 'documentLibrary' are backed-up.
this is in addition to 'webTemplateExtensionsList' templated lists which were previously skipped from restore.

the skipping of lists with template 'documentLibrary' is temporary and to be enabled with correct handling.

update:27/12/2023
skips documentLibrary, sharingLinks, webTemplateExtensionsList templated lists from being backed-up altogether

site contents of https://10rqc2.sharepoint.com/sites/CorsoCI:

backup details

./corso backup details sharepoint --backup 5e0a13b0-bff0-471b-8860-2aec869717b2
Logging to file: /home/hitesh/.cache/corso/logs/2023-12-23T14-52-37Z.log
Connecting to M365                               done 
Connecting to repository                     30s done 
  ID            List                                                     Items  Created               Modified            
  c7ffe2f0a275  Test List                                                0      2023-05-12T12:28:51Z  2023-05-12T12:28:51Z
  5345df87b565  Meh                                                      0      2023-05-12T12:59:50Z  2023-05-12T12:59:50Z
  cd2854591ab4  corso-restore-list_135b9067-0e19-4a5c-9475-ced51730c211  0      2023-12-11T10:25:27Z  2023-12-23T13:19:24Z
  0f56b7f450fa  corsorestorelist_newlist                                 0      2023-12-11T10:31:54Z  2023-12-11T10:31:54Z
  3a7be2fb9d50  new list                                                 0      2023-11-29T18:29:07Z  2023-11-29T18:29:07Z
  edab0c775e5e  New List Name                                            0      2023-05-12T12:39:06Z  2023-12-23T13:25:39Z
  4d7ad9b1df98  MockListing                                              0      2023-05-12T12:15:14Z  2023-12-14T06:22:36Z
  d2bdca596f75  integration-test-list                                    6      2023-11-30T06:57:01Z  2023-12-14T06:09:45Z
  c9eccc9771a3  Sharing Links                                            74     2023-06-26T07:21:31Z  2023-12-14T06:06:31Z
  5e79dcd93c92  New List Name 2                                          0      2023-05-12T12:40:15Z  2023-05-12T12:40:15Z
  6a6acbaa0d10  corsorestorelist_7f99d792-cae0-4a87-9e68-fd930f5b123e    0      2023-12-11T10:28:25Z  2023-12-23T13:19:57Z

restore details

./corso restore sharepoint --backup 5e0a13b0-bff0-471b-8860-2aec869717b2
Logging to file: /home/hitesh/.cache/corso/logs/2023-12-23T14-53-37Z.log
Connecting to M365                               done 
Connecting to repository                     26s done 
Restoring to folder Corso_Restore_23-Dec-2023_14-54-05
  
Restoring ∙ https://10rqc2.sharepoint.com/sites/CorsoCI              
Discovered 11 items in backup 5e0a13b0-bff0-471b-8860-2aec869717b2 to restore   
Enumerating items in repository             1m2s done 
Restoring data                               50s done 
Restore Complete
Restored 11 items
  ID            List                                                                                        Items  Created               Modified            
  edab0c775e5e  Corso_Restore_23-Dec-2023_14-54-05_New List Name                                            0      2023-12-23T14:54:23Z  2023-12-23T14:54:23Z
  cd2854591ab4  Corso_Restore_23-Dec-2023_14-54-05_corso-restore-list_135b9067-0e19-4a5c-9475-ced51730c211  0      2023-12-23T14:54:25Z  2023-12-23T14:54:25Z
  5345df87b565  Corso_Restore_23-Dec-2023_14-54-05_Meh                                                      0      2023-12-23T14:54:26Z  2023-12-23T14:54:26Z
  4d7ad9b1df98  Corso_Restore_23-Dec-2023_14-54-05_MockListing                                              0      2023-12-23T14:54:28Z  2023-12-23T14:54:28Z
  d2bdca596f75  Corso_Restore_23-Dec-2023_14-54-05_integration-test-list                                    6      2023-12-23T14:54:29Z  2023-12-23T14:54:31Z
  c7ffe2f0a275  Corso_Restore_23-Dec-2023_14-54-05_Test List                                                0      2023-12-23T14:54:34Z  2023-12-23T14:54:34Z
  c9eccc9771a3  Corso_Restore_23-Dec-2023_14-54-05_Sharing Links                                            74     2023-12-23T14:54:36Z  2023-12-23T14:54:36Z
  5e79dcd93c92  Corso_Restore_23-Dec-2023_14-54-05_New List Name 2                                          0      2023-12-23T14:55:09Z  2023-12-23T14:55:09Z
  6a6acbaa0d10  Corso_Restore_23-Dec-2023_14-54-05_corsorestorelist_7f99d792-cae0-4a87-9e68-fd930f5b123e    0      2023-12-23T14:55:10Z  2023-12-23T14:55:10Z
  0f56b7f450fa  Corso_Restore_23-Dec-2023_14-54-05_corsorestorelist_newlist                                 0      2023-12-23T14:55:11Z  2023-12-23T14:55:11Z

Does this PR need a docs update or release note?

⛔ No

Type of change

🐛 Bugfix

Issue(s)

#4754

Test Plan

💪 Manual
⚡ Unit test
💚 E2E

aviator-app · 2023-12-23T15:03:35Z

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This PR was merged using Aviator.

See the real-time status of this PR on the Aviator webapp.

Use the Aviator Chrome Extension to see the status of your PR within GitHub.

meain · 2023-12-26T10:47:30Z

src/internal/m365/collection/site/collection.go

@@ -358,7 +358,7 @@ func (sc *Collection) handleListItems(
 	rc := io.NopCloser(bytes.NewReader(entryBytes))
 	itemInfo := details.ItemInfo{
 		SharePoint:     info,
-		NotRecoverable: template == api.WebTemplateExtensionsListTemplateName,
+		NotRecoverable: api.NonRestorableListTemplates.HasKey(template),


I guess this could have been two ifs.

a list entry needs to be NonRecoverable if the template is either 'webTemplateExtensionsList' or 'documentLibrary'(temporarily). And NonRestorableListTemplates map contains both of them, so if the current listTemplate is either of the two types, NotRecoverable becomes true for that entry.

Make sense, I was mentioning that this could be template == api.WebTemplateExtensionsListTemplateName || template == api.documentLibrary. I personally feel this to be more clear.

meain · 2023-12-26T10:50:18Z

src/pkg/services/m365/api/lists.go

-		ptr.Val(newList.GetList().GetTemplate()) == WebTemplateExtensionsListTemplateName {
-		return nil, clues.StackWC(ctx, ErrCannotCreateWebTemplateExtension)
+		NonRestorableListTemplates.HasKey(ptr.Val(newList.GetList().GetTemplate())) {
+		return nil, clues.StackWC(ctx, ErrCannotCreateNonRestorableListTemplate)


I think the error could be something smaller like SystemListTempalte and we could have more information in the comments next to the const definition.

documentLibrary will not fall under system generated. Lists with that template needs to handled differently. Putting documentLibrary under non-restorable is temporary change. Should we shorten it to 'ErrNonRestorableListTemplate'

That sounds good, lets do ErrNonRestorableListTemplate.

src/pkg/services/m365/api/lists.go

sonarcloud · 2023-12-28T16:19:25Z

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

skips documentLibrary templated lists from restoration

ea8607d

HiteshRepo added restore sharepoint lists category of sharepoint service labels Dec 23, 2023

HiteshRepo self-assigned this Dec 23, 2023

HiteshRepo temporarily deployed to Testing December 23, 2023 15:03 — with GitHub Actions Inactive

HiteshRepo temporarily deployed to Testing December 23, 2023 15:04 — with GitHub Actions Inactive

HiteshRepo requested review from pandeyabs and meain December 23, 2023 16:29

meain reviewed Dec 26, 2023

View reviewed changes

resolve merge conflicts with main

80a5acf

HiteshRepo temporarily deployed to Testing December 26, 2023 13:33 — with GitHub Actions Inactive

HiteshRepo had a problem deploying to Testing December 26, 2023 13:33 — with GitHub Actions Error

HiteshRepo temporarily deployed to Testing December 26, 2023 13:33 — with GitHub Actions Inactive

HiteshRepo had a problem deploying to Testing December 26, 2023 13:33 — with GitHub Actions Failure

fix lint errors

3e96c15

HiteshRepo temporarily deployed to Testing December 26, 2023 13:44 — with GitHub Actions Inactive

HiteshRepo temporarily deployed to Testing December 27, 2023 10:49 — with GitHub Actions Inactive

HiteshRepo requested review from pandeyabs and meain December 27, 2023 11:14

pandeyabs approved these changes Dec 28, 2023

View reviewed changes

src/pkg/services/m365/api/lists.go Show resolved Hide resolved

adds comment for documentLibrary lists support

9437baf

HiteshRepo temporarily deployed to Testing December 28, 2023 05:03 — with GitHub Actions Inactive

HiteshRepo added the mergequeue label Dec 28, 2023

Merge branch 'main' into 4754-ignore-doclib

7209323

aviator-app bot temporarily deployed to Testing December 28, 2023 16:19 Inactive

aviator-app bot temporarily deployed to Testing December 28, 2023 16:20 Inactive

aviator-app bot merged commit 823fcdc into main Dec 28, 2023
24 checks passed

aviator-app bot deleted the 4754-ignore-doclib branch December 28, 2023 16:59

HiteshRepo mentioned this pull request Dec 29, 2023

Sharepoint Lists Backup & Restore #4754

Open

29 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

skips lists with certain templates from being backed-up #4931

skips lists with certain templates from being backed-up #4931

HiteshRepo commented Dec 23, 2023 •

edited

Loading

aviator-app bot commented Dec 23, 2023 •

edited

Loading

meain Dec 26, 2023

HiteshRepo Dec 26, 2023

meain Dec 27, 2023

meain Dec 26, 2023

HiteshRepo Dec 26, 2023 •

edited

Loading

meain Dec 27, 2023

sonarcloud bot commented Dec 28, 2023

skips lists with certain templates from being backed-up #4931

skips lists with certain templates from being backed-up #4931

Conversation

HiteshRepo commented Dec 23, 2023 • edited Loading

Does this PR need a docs update or release note?

Type of change

Issue(s)

Test Plan

aviator-app bot commented Dec 23, 2023 • edited Loading

Current Aviator status

meain Dec 26, 2023

Choose a reason for hiding this comment

HiteshRepo Dec 26, 2023

Choose a reason for hiding this comment

meain Dec 27, 2023

Choose a reason for hiding this comment

meain Dec 26, 2023

Choose a reason for hiding this comment

HiteshRepo Dec 26, 2023 • edited Loading

Choose a reason for hiding this comment

meain Dec 27, 2023

Choose a reason for hiding this comment

sonarcloud bot commented Dec 28, 2023

Quality Gate passed

HiteshRepo commented Dec 23, 2023 •

edited

Loading

aviator-app bot commented Dec 23, 2023 •

edited

Loading

HiteshRepo Dec 26, 2023 •

edited

Loading