Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support contents only for loki flusher #1256

Merged
merged 4 commits into from
Dec 22, 2023

Conversation

lujiajing1126
Copy link
Contributor

@lujiajing1126 lujiajing1126 commented Dec 5, 2023

Fix #1020 (comment)

  1. Add ContentsOnly option for loki flusher (currently only work for custom_single_flatten protocol)
  2. Polish docs (Fix [BUG]: ilogtail 将日志输入到 Loki 异常 #1020 (comment))

@CLAassistant
Copy link

CLAassistant commented Dec 5, 2023

CLA assistant check
All committers have signed the CLA.

@yyuuttaaoo yyuuttaaoo requested review from linrunqi08 and henryzhx8 and removed request for linrunqi08 December 15, 2023 03:49
Copy link
Collaborator

@henryzhx8 henryzhx8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TagKeyRenameMap将所有的tag作为key,空字符串作为value是否可以实现同样的效果?

@lujiajing1126
Copy link
Contributor Author

lujiajing1126 commented Dec 18, 2023

TagKeyRenameMap将所有的tag作为key,空字符串作为value是否可以实现同样的效果?

  1. 理论上你需要遍历所有的tag然后把它们全部置空;
  2. 同时会无法提取tag,
desiredValue, err := findTargetValues(targetFields, contents, tags, c.TagKeyRenameMap)

这一步提取的时候会根据TagKeyRenameMap填充值。如果把tag全部写成空值,在这一步会拿不到任何tag。在loki的场景下,需要把部分的tag指定成stream label,所以这种方式不太可行。

@henryzhx8
Copy link
Collaborator

TagKeyRenameMap将所有的tag作为key,空字符串作为value是否可以实现同样的效果?

  1. 理论上你需要遍历所有的tag然后把它们全部置空;
  2. 同时会无法提取tag,
desiredValue, err := findTargetValues(targetFields, contents, tags, c.TagKeyRenameMap)

这一步提取的时候会根据TagKeyRenameMap填充值。如果把tag全部写成空值,在这一步会拿不到任何tag。在loki的场景下,需要把部分的tag指定成stream label,所以这种方式不太可行。

了解,你实际上只是想实现发送环节的only content

@lujiajing1126
Copy link
Contributor Author

TagKeyRenameMap将所有的tag作为key,空字符串作为value是否可以实现同样的效果?

  1. 理论上你需要遍历所有的tag然后把它们全部置空;
  2. 同时会无法提取tag,
desiredValue, err := findTargetValues(targetFields, contents, tags, c.TagKeyRenameMap)

这一步提取的时候会根据TagKeyRenameMap填充值。如果把tag全部写成空值,在这一步会拿不到任何tag。在loki的场景下,需要把部分的tag指定成stream label,所以这种方式不太可行。

了解,你实际上只是想实现发送环节的only content

对的。我想那个issue应该也是这个意思,这个确实是和loki的具体用法有关。为了不破坏ilogtail的整体逻辑,只能这么做

@yyuuttaaoo yyuuttaaoo merged commit c81b73e into alibaba:main Dec 22, 2023
19 checks passed
@yyuuttaaoo yyuuttaaoo added the enhancement Feature enhancement label Dec 22, 2023
@lujiajing1126 lujiajing1126 deleted the support-contents-only branch December 22, 2023 09:05
@yyuuttaaoo yyuuttaaoo added this to the v2.0 milestone Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG]: ilogtail 将日志输入到 Loki 异常
4 participants