Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker container name not present in docker container logs. #1597

Closed
nityanandagohain opened this issue Sep 27, 2022 · 5 comments
Closed

Docker container name not present in docker container logs. #1597

nityanandagohain opened this issue Sep 27, 2022 · 5 comments
Assignees
@nityanandagohain
Labels
request-more-info

Comments

@nityanandagohain
Copy link
Member

nityanandagohain commented Sep 27, 2022 •
edited
Loading

Right now the default docker log collection doesn't have the container name in attributes.
@nityanandagohain nityanandagohain self-assigned this Sep 27, 2022
@request-info
Copy link

request-info bot commented Sep 27, 2022

We would appreciate it if you could provide us with more info about this issue/pr!

@nityanandagohain
Copy link
Member Author

nityanandagohain commented Sep 27, 2022 •
edited
Loading

Solution using logsprout .

We can route the logs to signoz otel collector from logsprout and we can get the container id as well.

The configuration for it is

receivers:
  .........
  tcplog:
    listen_address: "0.0.0.0:2255"
    attributes: {}
    add_attributes: false
    operators:
      - type: regex_parser
        regex: '^<(?P<priority>[0-9]+)>[0-9]+ (?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}(\.[0-9]+)?([zZ]|([\+-])([01]\d|2[0-3]):?([0-5]\d)?)?) (?P<container_id>\S+) (?P<container_name>\S+) [0-9]+ - - (?P<body>.*)'
      - type: move
        from: attributes["body"]
        to: body
      - type: time_parser
        parse_from: attributes.timestamp
        # layout: '%Y-%m-%dT%H:%M:%SZ'
        layout: '%Y-%m-%dT%H:%M:%S.%LZ'
      - type: remove
        field: attributes.timestamp
.....
service:
  pipelines:
    .........
    logs:
      receivers: [tcplog, otlp]
      processors: [batch]
      exporters: [clickhouselogsexporter]

We will also need to export 2255 port for otel collector.

signoz/deploy/docker/clickhouse-setup/docker-compose.yaml

Line 207 in a3c917c

# - "1777:1777" # pprof extension

Now we can start logspout by running

docker run --net=host --rm --name="logspout" \
        --volume=/var/run/docker.sock:/var/run/docker.sock \
        gliderlabs/logspout \
        syslog+tcp://0.0.0.0:2255

We will be able to see the attributes in the UI

Screenshot 2022-09-27 at 2 44 52 PM

@nityanandagohain
Copy link
Member Author

@ankitnayan should we turn this into a tutorial/blog.

@nityanandagohain
Copy link
Member Author

nityanandagohain commented Jan 30, 2023 •
edited
Loading

Closing this, as there is no more actionable here. The above solution with the logsprout works.

@rwjack rwjack mentioned this issue Feb 27, 2023
Closed
@nityanandagohain nityanandagohain mentioned this issue Jun 7, 2023
Closed
@Alkaponees
Copy link

Hello i want ot show container_id from 12 symbols in signoz but idk how to do it.
I use default config for collecting docker container logs. Could you help me with this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nityanandagohain nityanandagohain

Labels
request-more-info
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nityanandagohain @Alkaponees