Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source: fluent, allow sending events to unix socket #17132

Open
nhlushak opened this issue Apr 12, 2023 · 3 comments · May be fixed by #22212
Open

Source: fluent, allow sending events to unix socket #17132

nhlushak opened this issue Apr 12, 2023 · 3 comments · May be fixed by #22212
Labels
source: fluent Anything `fluent` source related type: feature A value-adding code addition that introduce new functionality.

Comments

@nhlushak
Copy link

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

We are activelly using Vector as a replacement for FluentD in our ECS Fargate. We successfully utilize logging libraries in our code that are wrapping events via forward protocol and sending them to tcp endpoint in Vector.

But now we are looking towards using Vector sidecar as ECS Firelens container to receive console logs from main containers via fluent logging driver. Unfortunately, sending logs to Firelens container is available only via Unix socket which Vector is currently is not able to do.

Attempted Solutions

We tried using socket source but this is not the case, since data sent in Fluent format.

Current workaround we achieved is to use splunk logging driver and splunk hec source. That is working, but we lack some addtitional info that ECS provides for log events sent through Firelens.

Proposal

Extend fluent source so it could receive data not only through adress (e.g. localhost:24224) but also via UNIX socket (e.g. /var/run/fluent.sock).

References

No response

Version

0.28.2

@nhlushak nhlushak added the type: feature A value-adding code addition that introduce new functionality. label Apr 12, 2023
@neuronull neuronull added the provider: confluent Anything `confluent` service provider related label Apr 12, 2023
@zamazan4ik
Copy link
Contributor

zamazan4ik commented Apr 13, 2023

We tried using socket source but this is not the case, since data sent in Fluent format.

Hmmm. These words sound like we need to implement fluent codec... With that, using socket source with fluent codec should be ok.

@nhlushak
Copy link
Author

@zamazan4ik This might be a solution, indeed.

@jszwedko jszwedko added source: fluent Anything `fluent` source related and removed provider: confluent Anything `confluent` service provider related labels Apr 13, 2023
@mrzor
Copy link

mrzor commented Apr 13, 2023

socat could help bridging a unix socket to Vector while you wait for a proper fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
source: fluent Anything `fluent` source related type: feature A value-adding code addition that introduce new functionality.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants