Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VMAgent] source_labels or sourceLabels | Operator claims spec.remoteWrite in managedFields #946

Closed
sdomme opened this issue May 6, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@sdomme
Copy link

sdomme commented May 6, 2024

Hi,

I have a question regarding using the source_labels/sourceLabels and/or target_label/targetLabel in a spec.remoteWrite inlineUrlRelabelConfig (probably global RelabelConfig is also affected).

version: victoriametrics/operator:v0.42.3

It doesn't matter if you deploy a VMAgent with the underscore version for inlineUrlRelabelConfig like this:

- inlineUrlRelabelConfig:
  - action: replace
    regex: vmagent_remotewrite_bytes_sent_total
    replacement: new_label_value
    source_labels:
      - __name__
    target_label: new_label
  url: https://url.xyz

or the version without underscore like this:

- inlineUrlRelabelConfig:
  - action: replace
    regex: vmagent_remotewrite_bytes_sent_total
    replacement: new_label_value
    sourceLabels:
      - __name__
    targetLabel: new_label
  url: https://url.xyz

it ends up building this:

- inlineUrlRelabelConfig:
  - action: replace
    regex: vmagent_remotewrite_bytes_sent_total
    replacement: new_label_value
    sourceLabels:
      - __name__
    source_labels:
      - __name__
    targetLabel: new_label
    target_label: new_label
  url: https://url.xyz

The issue we have now is, in any case the operator claims the spec.remoteWrite field in managedFields:

managedFields:
- ...
- apiVersion: operator.victoriametrics.com/v1beta1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            f:operator.victoriametrics/last-applied-spec: {}
          f:finalizers:
            .: {}
            v:"apps.victoriametrics.com/finalizer": {}
        f:spec:
          f:remoteWrite: {}
      manager: manager
      operation: Update
      time: '2024-05-06T13:35:58Z'

that leads to an apply conflict for remoteWrite changes as per this definition for the manager that handled the initial creation of the VMAgent object with a Server-Side Apply.

│ Error: victoria/application failed to run apply: Apply failed with 1 conflict: conflict with "manager" using operator.victoriametrics.com/v1beta1: .spec.remoteWrite
│ Please review the fields above--they currently have other managers. Here
│ are the ways you can resolve this warning:
│ * If you intend to manage all of these fields, please re-run the apply
│   command with the `--force-conflicts` flag.
│ * If you do not intend to manage all of the fields, please edit your
│   manifest to remove references to the fields that should keep their
│   current managers.
│ * You may co-own fields by updating your manifest to match the existing
│   value; in this case, you'll become the manager if the other manager(s)
│   stop managing the field (remove it from their configuration).
│ See http://k8s.io/docs/reference/using-api/server-side-apply/#conflicts

I can't remember having this issue before, but we also didn't change remoteWrite for a longer time.

Also existing VMAgents aren't affected by this. The operator does not set this double version parameters. Only if we create new ones.

@f41gh7 f41gh7 added the bug Something isn't working label May 7, 2024
@f41gh7
Copy link
Collaborator

f41gh7 commented May 7, 2024

Hello, thanks for reporting. It's a bug, we're going to fix it at the next release.

f41gh7 added a commit that referenced this issue May 7, 2024
It must possible issues with JSON marshalling mutations and remove default values setting for CRD object.

Previously, it was possible, that object was modified during unmarshalling, it causes object mutation after client.Update call. Since object state wasn't restored to the original version after marshalling.

#946
f41gh7 added a commit that referenced this issue May 7, 2024
…ate (#950)

It must possible issues with JSON marshalling mutations and remove default values setting for CRD object.

Previously, it was possible, that object was modified during unmarshalling, it causes object mutation after client.Update call. Since object state wasn't restored to the original version after marshalling.

#946
@f41gh7
Copy link
Collaborator

f41gh7 commented May 10, 2024

Changes included to the v0.44.0 release

Feel free to re-open if it doesnt help

@f41gh7 f41gh7 closed this as completed May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants