Skip to content

Retrieve the VolumeSnapshotContent before attempting an update#524

Closed
huffmanca wants to merge 1 commit intokubernetes-csi:masterfrom
huffmanca:retrieve-vsc-before-update
Closed

Retrieve the VolumeSnapshotContent before attempting an update#524
huffmanca wants to merge 1 commit intokubernetes-csi:masterfrom
huffmanca:retrieve-vsc-before-update

Conversation

@huffmanca
Copy link
Contributor

What type of PR is this?
/kind bug

What this PR does / why we need it:
It's been noticed that when we attempt to update the VSC's error the following error can be thrown:

I0525 13:07:49.293034       1 snapshot_controller.go:165] updating VolumeSnapshotContent[snapcontent-f24fd046-1664-4bb2-bb43-f91ae0909b02] error status failed Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io "snapcontent-f24fd046-1664-4bb2-bb43-f91ae0909b02": the object has been modified; please apply your changes to the latest version and try again

This PR forces a Get before we attempt the update so that we can avoid this error.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 25, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: huffmanca
To complete the pull request process, please assign jingxu97 after the PR has been reviewed.
You can assign the PR to them by writing /assign @jingxu97 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label May 25, 2021
@ggriffiths
Copy link
Contributor

These seems to cut the number of errors in half from 56 to 28:

with this change:

~ curl -s https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_external-snapshotter/524/pull-kubernetes-csi-external-snapshotter-1-21-on-kubernetes-1-21/1397246208530976768/artifacts/kube-system/snapshot-controller-58f54d8f6c-7t6tx/snapshot-controller.log | grep 'the object has been modified; please apply your' | wc -l
      28

without:

~ curl -s https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_external-snapshotter/519/pull-kubernetes-csi-external-snapshotter-1-21-on-kubernetes-1-21/1396873247164731392/artifacts/kube-system/snapshot-controller-58f54d8f6c-9gt8c/snapshot-controller.log |  grep 'the object has been modified; please apply your' | wc -l
      56

I believe there was a concern about excessive API calls if we get the latest object before every update, but this will only pull the latest object for updateContentErrorStatusWithEvent. I think it's okay to add this change as a temporary improvement until a permanent fix is added. Thoughts on this @xing-yang ?

@huffmanca huffmanca force-pushed the retrieve-vsc-before-update branch from 9817235 to 57cbe6f Compare May 26, 2021 12:36
@xing-yang
Copy link
Collaborator

I believe there was a concern about excessive API calls if we get the latest object before every update, but this will only pull the latest object for updateContentErrorStatusWithEvent. I think it's okay to add this change as a temporary improvement until a permanent fix is added. Thoughts on this @xing-yang ?

I still have this concern. Let's discuss about this in the CSI meeting.

@huffmanca
Copy link
Contributor Author

I was watching #480 to see if it would get merged, but after it was closed I submitted this PR in the hopes of addressing one of the issues. Right now we can encounter a state where an error isn't added to the VSC due to constant object has been modified errors.

@huffmanca huffmanca force-pushed the retrieve-vsc-before-update branch from 57cbe6f to 022726c Compare May 26, 2021 14:09
@huffmanca huffmanca force-pushed the retrieve-vsc-before-update branch from 022726c to 9ae124e Compare May 26, 2021 14:10
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 24, 2021
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Sep 23, 2021
@k8s-ci-robot
Copy link
Contributor

@huffmanca: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 6, 2021
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue or PR with /reopen
  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

@k8s-ci-robot
Copy link
Contributor

@k8s-triage-robot: Closed this PR.

Details

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue or PR with /reopen
  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants