Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent results when using logfmt parser stage with fields #15190

Open
chaudum opened this issue Nov 29, 2024 · 0 comments
Open

Inconsistent results when using logfmt parser stage with fields #15190

chaudum opened this issue Nov 29, 2024 · 0 comments
Labels
type/bug Somehing is not working as expected

Comments

@chaudum
Copy link
Contributor

chaudum commented Nov 29, 2024

Queries that aggregate on extracted labels (sum by (label_a, label_b)) return different results when using | logfmt and | logfmt label_a, label_b.

Example queries:

$ logcli instant-query 'sum by (err, msg) (count_over_time({**********} |= "level=error" | logfmt [24h]))'  | jq . 

[
  {
    "metric": {
      "err": "**********",
      "msg": "failed to do request"
    },
    "value": [
      1732717258.183,
      "2"
    ]
  },
  {
    "metric": {
      "err": "**********",
      "msg": "backoff terminated unsuccessfully"
    },
    "value": [
      1732717258.183,
      "1"
    ]
  },
  {
    "metric": {
      "msg": "Error in comparison client"
    },
    "value": [
      1732717258.183,
      "1"
    ]
  }
]

vs

$ logcli instant-query 'sum by (err, msg) (count_over_time({**********} |= "level=error" | logfmt err, msg [24h]))'  | jq .

[
  {
    "metric": {
      "msg": "Error in comparison client"
    },
    "value": [
      1732717323.361,
      "1"
    ]
  },
  {
    "metric": {
      "msg": "backoff terminated unsuccessfully"
    },
    "value": [
      1732717323.361,
      "1"
    ]
  },
  {
    "metric": {
      "msg": "failed to do request"
    },
    "value": [
      1732717323.361,
      "2"
    ]
  }
]

The second query, where err and msg are part of the logfmt parser stage, is missing the err label in the series of the result.

When adding another label, which does not even need to exist, to the aggregation, the err label becomes part of the series again:

$ logcli instant-query 'sum by (err, msg, foo) (count_over_time({**********} |= "level=error" | logfmt err,msg [24h]))'  | jq .

[
  {
    "metric": {
      "err": "**********",
      "msg": "failed to do request"
    },
    "value": [
      1732717574.53,
      "2"
    ]
  },
  {
    "metric": {
      "err": "**********",
      "msg": "backoff terminated unsuccessfully"
    },
    "value": [
      1732717574.53,
      "1"
    ]
  },
  {
    "metric": {
      "msg": "Error in comparison client"
    },
    "value": [
      1732717574.53,
      "1"
    ]
  }
]
@chaudum chaudum added the type/bug Somehing is not working as expected label Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Somehing is not working as expected
Projects
None yet
Development

No branches or pull requests

1 participant