-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
parses metadata collections using previous paths #4947
Conversation
Current Aviator status
This PR was merged manually (without Aviator). Merging manually can negatively impact the performance of the queue. Consider using Aviator next time. See the real-time status of this PR on the Aviator webapp. Use the Aviator Chrome Extension to see the status of your PR within GitHub.
|
return nil, false, err | ||
} | ||
|
||
// [TODO] utilise deltapaths to determine list's state |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: We use // TODO(hiteshrepo): comment
as a convention for marking todos in code.
} | ||
|
||
// [TODO] utilise deltapaths to determine list's state | ||
_ = dps |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: You could do this in line 163 itself i.e. _, canUsePreviousBackup, err
"github.com/alcionai/corso/src/pkg/path" | ||
) | ||
|
||
func parseMetadataCollections( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
This is missing a unit test. You can use
TestParseMetadataCollections
as a reference. -
Also, should we make this function
list
specific? Maybe call the filelist_metadata.go
as well. Reasons:
- It can be confusing since it comes off as a generic func used for all
sites
categories. But it's only being used for lists. - It doesn't handle delta file paths, which is a list behavior. Might not be true for other sites categories. Please correct me if wrong.
- Could you please also file a tech debt issue to make
parseMetadataCollections
a generic function which works for all services? Right now this code is duplicated everywhere, but most of it is common for all services. We don't need to fix the tech debt right away, but it'd be good to track it as an issue. Let me know if you disagree :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Yeah sure I will write the unit test for it.
- I thought, when sharepoint page is being implemented, that would to take care of dividing/unifying the function. But lets do what you suggested. In that case while page is being implemented, we have to consider unifying the logic based on delta path availability for page.
- Yeah. I will assimilate tech-debts arising from PRs in the notion page momentarily and then move them to appropriate location (GH/linear).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving to unblock as the remaining PRs in the chain look good in general.
Please feel free to add new PRs for missing test coverages if that makes it more convenient for you.
refactors code and tests site populatecollection This is the final PR of the support of 'previouspath' for sharepoint lists PR chain: - #4938 - #4946 - #4947 - #4948 - #4950 Tested manually if the deleted list in the site gets reflected in the next backup or not and here are the results: **Backup#1** Site has a list named `temp-list` with list id `d5b9b13e-4646-4374-902e-7c3c2cd4501d`. lists stored in kopia (11 entries): ``` drwxrwxrwx 18959 2023-12-27 22:27:09 IST k12cbdb5cc1272f21a4bd89b80865b059 MjJlNjA5YWMtZTU0OC00MzY1LTk4ZTctM2IyYWY5Mjg5MDg2/ drwxrwxrwx 20937 2023-12-23 19:17:59 IST kb6b9f232cc3e12a4e6d409472adfa16a NTVkMTkwY2QtNTEwOS00NzljLThlNWQtZjg4YzczNjYzMjJh/ drwxrwxrwx 17152 2023-12-14 13:00:35 IST k497bcade330714d9e319719cb69719f0 NzU0YWY3YTQtNjgyZi00MGU2LThiMGEtOWNhYTRhYzYyOGJl/ drwxrwxrwx 18285 2023-12-10 11:40:54 IST kdb724b7af6812fb00790aa9de922de85 ZGE3NmY5OGYtYjYyMS00YTU3LThjNDktYTkxNzE4ZTM4YmRm/ drwxrwxrwx 15714 2023-12-14 13:00:29 IST kedba09b1ec8dfda71fe2783a29550b7f ZThkZjY5YWQtZDM1Ni00ZWI4LTkyMTEtZWY3Nzc2YTIxMzVk/ drwxrwxrwx 22938 2023-12-14 13:00:29 IST k3e9b6a6a02a1269839141464efb7c82b ZjgyZjkwMGItZmM5YS00Mzc2LTlkYmMtODkwOTA5YjNhMzAy/ drwxrwxrwx 17311 2023-12-27 22:20:56 IST k89708dca3aed5d2d46bbc4aa74866cd4 MDhhMTFhZTMtZDg0ZS00ZWYxLTg4M2ItOTA5OTcyOGExYzEx/ drwxrwxrwx 17289 2023-12-28 16:36:10 IST k9fb16c9fde760b9f1e4e59a6f5b4ad2a MTMxNDdhODctZmExNS00OTdlLTgzNTEtYmY5YjJiYTQwMWRi/ drwxrwxrwx 15678 2023-12-14 13:00:29 IST kcf4669bb29e8bf34e1a93c3c7126568d Mzc0OTkxYzItOTE1NC00ZjdlLWI1ZjUtOTUzMjQ2MTAwMjFh/ drwxrwxrwx 15655 2023-12-14 13:00:30 IST ka887f4845d704ffb7d104c88535cf278 NDIwNzNmNGQtZTYwNS00ZmFkLTg4ZTAtNjZiMTcyNmExODM5/ drwxrwxrwx 14570 2023-12-29 13:05:03 IST kc8d1a0a5d2c0e53305e88c4f692f3e97 ZDViOWIxM2UtNDY0Ni00Mzc0LTkwMmUtN2MzYzJjZDQ1MDFk/ --> temp-list (d5b9b13e-4646-4374-902e-7c3c2cd4501d) ``` previous paths stored in kopia (11 entries): ``` {"08a11ae3-d84e-4ef1-883b-9099728a1c11":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/08a11ae3-d84e-4ef1-883b-9099728a1c11","13147a87-fa15-497e-8351-bf9b2ba401db":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/13147a87-fa15-497e-8351-bf9b2ba401db","22e609ac-e548-4365-98e7-3b2af9289086":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/22e609ac-e548-4365-98e7-3b2af9289086","374991c2-9154-4f7e-b5f5-95324610021a":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/374991c2-9154-4f7e-b5f5-95324610021a","42073f4d-e605-4fad-88e0-66b1726a1839":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/42073f4d-e605-4fad-88e0-66b1726a1839","55d190cd-5109-479c-8e5d-f88c7366322a":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/55d190cd-5109-479c-8e5d-f88c7366322a","754af7a4-682f-40e6-8b0a-9caa4ac628be":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/754af7a4-682f-40e6-8b0a-9caa4ac628be","d5b9b13e-4646-4374-902e-7c3c2cd4501d":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/d5b9b13e-4646-4374-902e-7c3c2cd4501d","da76f98f-b621-4a57-8c49-a91718e38bdf":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/da76f98f-b621-4a57-8c49-a91718e38bdf","e8df69ad-d356-4eb8-9211-ef7776a2135d":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/e8df69ad-d356-4eb8-9211-ef7776a2135d","f82f900b-fc9a-4376-9dbc-890909b3a302":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/f82f900b-fc9a-4376-9dbc-890909b3a302"} ``` **Backup#2** Deleted list `temp-list` from the site lists stored in kopia (10 entries): ``` drwxrwxrwx 15678 2023-12-14 13:00:29 IST kcf4669bb29e8bf34e1a93c3c7126568d Mzc0OTkxYzItOTE1NC00ZjdlLWI1ZjUtOTUzMjQ2MTAwMjFh/ drwxrwxrwx 20937 2023-12-23 19:17:59 IST kb6b9f232cc3e12a4e6d409472adfa16a NTVkMTkwY2QtNTEwOS00NzljLThlNWQtZjg4YzczNjYzMjJh/ drwxrwxrwx 17152 2023-12-14 13:00:35 IST k497bcade330714d9e319719cb69719f0 NzU0YWY3YTQtNjgyZi00MGU2LThiMGEtOWNhYTRhYzYyOGJl/ drwxrwxrwx 18285 2023-12-10 11:40:54 IST kdb724b7af6812fb00790aa9de922de85 ZGE3NmY5OGYtYjYyMS00YTU3LThjNDktYTkxNzE4ZTM4YmRm/ drwxrwxrwx 15714 2023-12-14 13:00:29 IST kedba09b1ec8dfda71fe2783a29550b7f ZThkZjY5YWQtZDM1Ni00ZWI4LTkyMTEtZWY3Nzc2YTIxMzVk/ drwxrwxrwx 17311 2023-12-27 22:20:56 IST k89708dca3aed5d2d46bbc4aa74866cd4 MDhhMTFhZTMtZDg0ZS00ZWYxLTg4M2ItOTA5OTcyOGExYzEx/ drwxrwxrwx 17289 2023-12-28 16:36:10 IST k9fb16c9fde760b9f1e4e59a6f5b4ad2a MTMxNDdhODctZmExNS00OTdlLTgzNTEtYmY5YjJiYTQwMWRi/ drwxrwxrwx 18959 2023-12-27 22:27:09 IST k12cbdb5cc1272f21a4bd89b80865b059 MjJlNjA5YWMtZTU0OC00MzY1LTk4ZTctM2IyYWY5Mjg5MDg2/ drwxrwxrwx 22938 2023-12-14 13:00:29 IST k3e9b6a6a02a1269839141464efb7c82b ZjgyZjkwMGItZmM5YS00Mzc2LTlkYmMtODkwOTA5YjNhMzAy/ drwxrwxrwx 15655 2023-12-14 13:00:30 IST ka887f4845d704ffb7d104c88535cf278 NDIwNzNmNGQtZTYwNS00ZmFkLTg4ZTAtNjZiMTcyNmExODM5/ ``` previous paths stored in kopia (10 entries): ``` {"08a11ae3-d84e-4ef1-883b-9099728a1c11":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/08a11ae3-d84e-4ef1-883b-9099728a1c11","13147a87-fa15-497e-8351-bf9b2ba401db":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/13147a87-fa15-497e-8351-bf9b2ba401db","22e609ac-e548-4365-98e7-3b2af9289086":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/22e609ac-e548-4365-98e7-3b2af9289086","374991c2-9154-4f7e-b5f5-95324610021a":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/374991c2-9154-4f7e-b5f5-95324610021a","42073f4d-e605-4fad-88e0-66b1726a1839":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/42073f4d-e605-4fad-88e0-66b1726a1839","55d190cd-5109-479c-8e5d-f88c7366322a":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/55d190cd-5109-479c-8e5d-f88c7366322a","754af7a4-682f-40e6-8b0a-9caa4ac628be":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/754af7a4-682f-40e6-8b0a-9caa4ac628be","da76f98f-b621-4a57-8c49-a91718e38bdf":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/da76f98f-b621-4a57-8c49-a91718e38bdf","e8df69ad-d356-4eb8-9211-ef7776a2135d":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/e8df69ad-d356-4eb8-9211-ef7776a2135d","f82f900b-fc9a-4376-9dbc-890909b3a302":"fb8afbaa-e94c-4ea5-8a8a-24aff04d7874/sharepoint/10rqc2.sharepoint.com,72b1468e-7a6c-4622-bf83-0e2e8f009afe,2c6a13db-cf82-41d4-ab21-0d5d4ecb4481/lists/f82f900b-fc9a-4376-9dbc-890909b3a302"} ``` #### Does this PR need a docs update or release note? - [x] :no_entry: No #### Type of change <!--- Please check the type of change your PR introduces: ---> - [x] :robot: Supportability/Tests #### Issue(s) #4754 #### Test Plan <!-- How will this be tested prior to merging.--> - [x] πͺ Manual - [x] β‘ Unit test - [x] π E2E
determines sharepoint collection state using current and previous paths #### Does this PR need a docs update or release note? - [x] β No #### Type of change <!--- Please check the type of change your PR introduces: ---> - [x] π» Feature #### Issue(s) #4754 #### Test Plan <!-- How will this be tested prior to merging.--> - [x] πͺ Manual - [x] β‘ Unit test - [x] π E2E
Quality Gate passedThe SonarCloud Quality Gate passed, but some issues were introduced. 4 New issues |
parses metadata collections using previous paths
Does this PR need a docs update or release note?
Type of change
Issue(s)
#4754
Test Plan