Sharepoint Lists Backup & Restore

[HiteshRepo]

Sharepoint support is not enabled yet
Backup Disabled:

• disable backup of sharepoint lists #4960

• corso/src/cli/utils/sharepoint.go

  Line 72 in b49c124

  // flags.DataLists: {}, [TODO]: uncomment when lists are enabled

  
  Restore Disabled:

  corso/src/cli/restore/sharepoint.go

  Line 99 in 13f549c

  sel.Exclude(sel.Lists(selectors.Any()))

  
  Export Disabled:

  corso/src/cli/export/sharepoint.go

  Line 99 in 13f549c

  sel.Exclude(sel.Lists(selectors.Any()))

Action Items

---

Enumeration - #4917

• All lists
• All list items
• All columns
• All contentTypes
• All columnLinks
• All contentTypeColumns

Interfaces - #4916

• ListInfo - stores List related details like Name, Template, Item count, Modified time and Created time
• SharepointInfo - to accommodate ListInfo, give appropriate headers and values

APIs - #4917

Package: src/pkg/services/m365/api/lists.go & src/pkg/services/m365/api/lists_pager.go

• GetLists - to fetch all lists (call config to request list-id and last modified time)
• GetListByID - to fetch a list by id, also make requests to fetch associated relations (columns, contentTypes) and list items
• PostList - to create a new list from stored list bytes
• DeleteList - to delete a list

Packages

• internal/m365/collection/site - to accommodate lists backup and restore handlers - adds list backup and restore handlers #4919
• internal/m365/collection/site/backup.go - creates new collections from fetching all lists - enables sharepoint to use lists backup handler for lists ops #4920
• internal/m365/collection/site/collection.go - updates to use list backup handler to fetch list by id and stream items for all fetched collections - enables sharepoint to use lists backup handler for lists ops #4920
• internal/m365/collection/site/restore.go - creates stored list for a particular backup - enables sharepoint to use lists restore handler for lists ops #4923
• internal/m365/collection/site/collection.go - accommodates lazyFetchCollection and preFetchCollection based on valid last modified time for Kopia Assist backup enablement - lists prefetech #4943

Others

• Enable restore path creation - enable restore path creation for sharepoint lists #4922
• CLI level changes - cli changes for sharepoint lists #4924
• Creates previous path file for state determination and tombstone deleted collections - refactors code and tests site populatecollection #4951
• Storage of metadata separately
• Incremental support for lists based last modified time
• Sanity tests --> adds CI sanity test steps to github workflow #5037
• Store permissisons for lists, requires exploration --> [Update 1/1/2024]: Not required
• Advanced restores --> handles lists restore collision #5026
• Backup and restore attachments --> [Update 16/1/2024]: Raise a known issue

Pending issues

• Handle hyperlink & Location columns/items - Hyperlink, Location
• Render choice columns/items properly
• Add fault bus for recoverable list items craetion errors and remove rollback of list - replaces deleteList with errors fault bus #4933
• Skip backup of lists with template documentLibrary, sharingLinks, accessRequest and pageLibrary - Issue, PR, PR-accessRequest

Proposal & Analysis

• Whole Proposal
• Check if metadata segregation is required
• Advanced Restores