Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

systemd: build: Add sethostname workaround for AmazonLinux2 #9845

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Jan 17, 2025

In AmazonLinux2, it is built on top of coreos technology. So, this race condition bug is also existing there.
In this PR, I added a workaround which is proposed in #9506 (comment).

This workaround is only effective for AmazonLinux2.

Closes #9506


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
[SERVICE]
    flush        1
    daemon       Off
    log_level    info
    http_server  Off
    http_listen  0.0.0.0
    http_port    2020
    storage.metrics on

[INPUT]
    name dummy
    tag  dummy.local

    # Read interval (sec) Default: 1
    interval_sec 10

[FILTER]
    Name  record_modifier
    Match *
    Record hostname ${HOSTNAME}
    Record filter_type record_modifiler

[OUTPUT]
    name  stdout
    match *
  • Debug log output from testing the change

It's log for confirming on self-created AMI on AWS EC2 instance.

[ec2-user@ip-172-31-17-102 ~]$ systemctl status fluent-bit -l
● fluent-bit.service - Fluent Bit
   Loaded: loaded (/usr/lib/systemd/system/fluent-bit.service; enabled; vendor preset: disabled)
   Active: active (running) since 月 2025-01-20 08:07:20 UTC; 52s ago
     Docs: https://docs.fluentbit.io/manual/
 Main PID: 1034 (fluent-bit)
   CGroup: /system.slice/fluent-bit.service
           └─1034 /opt/fluent-bit/bin/fluent-bit -c //etc/fluent-bit/fluent-bit.conf

 1月 20 08:07:20 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [2025/01/20 08:07:20] [ info] [input:dummy:dummy.0] initializing
 1月 20 08:07:20 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [2025/01/20 08:07:20] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
 1月 20 08:07:20 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [2025/01/20 08:07:20] [ info] [sp] stream processor started
 1月 20 08:07:20 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [2025/01/20 08:07:20] [ info] [output:stdout:stdout.0] worker #0 started
 1月 20 08:07:30 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [0] dummy.local: [[1737360449.913292106, {}], {"message"=>"dummy", "hostname"=>"ip-172-31-17-102.ap-northeast-1.compute.internal", "filter_type"=>"record_modifiler"}]
 1月 20 08:07:40 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [0] dummy.local: [[1737360459.913322822, {}], {"message"=>"dummy", "hostname"=>"ip-172-31-17-102.ap-northeast-1.compute.internal", "filter_type"=>"record_modifiler"}]
 1月 20 08:07:50 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [0] dummy.local: [[1737360469.913313391, {}], {"message"=>"dummy", "hostname"=>"ip-172-31-17-102.ap-northeast-1.compute.internal", "filter_type"=>"record_modifiler"}]
 1月 20 08:08:00 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [0] dummy.local: [[1737360479.913316966, {}], {"message"=>"dummy", "hostname"=>"ip-172-31-17-102.ap-northeast-1.compute.internal", "filter_type"=>"record_modifiler"}]
 1月 20 08:08:10 ip-172-31-17-102.ap-northeast-1.compute.internal fluent-bit[1034]: [0] dummy.local: [[1737360489.913314708, {}], {"message"=>"dummy", "hostname"=>"ip-172-31-17-102.ap-northeast-1.compute.internal", "filter_type"=>"record_modifiler"}]
[ec2-user@ip-172-31-17-102 ~]$ hostname
ip-172-31-17-102.ap-northeast-1.compute.internal

Plus, the packer template for creating AMI:

packer {
  required_plugins {
    amazon = {
      version = ">= 1.2.8"
      source  = "github.com/hashicorp/amazon"
    }
  }
}

source "amazon-ebs" "amazonlinux2" {
  ami_name      = "fbit-amazon-linux2-arm64"
  instance_type = "t4g.micro"
  region        = "ap-northeast-1"
  source_ami_filter {
    filters = {
      name                             = "*amzn2-ami-hvm-*"
      root-device-type                 = "ebs"
      virtualization-type              = "hvm"
      architecture                     = "arm64"
    }
    most_recent = true
    owners      = ["amazon"]
  }
  ssh_username = "ec2-user"
}

build {
  name    = "az2-fbit-packer"
  sources = [
    "source.amazon-ebs.amazonlinux2"
  ]

  provisioner "file" {
    source = "/path/to/fluent-bit/packaging/packages/amazonlinux/2/2025-01-20-15_52_36/fluent-bit-4.0.0-1.aarch64.rpm"
    destination = "/home/ec2-user/fluent-bit.rpm"
  }

  provisioner "file" {
    source = "/path/to/GitHub/aws_az2_ami/fluent-bit.conf"
    destination = "/home/ec2-user/fluent-bit.conf"
  }

  provisioner "shell" {
    inline = [
        "sudo yum install -y /home/ec2-user/fluent-bit.rpm",
        "sudo mv /home/ec2-user/fluent-bit.conf /etc/fluent-bit/fluent-bit.conf",
        "sudo systemctl enable fluent-bit"
    ]
  }
}
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@cosmo0920 cosmo0920 added the ok-package-test Run PR packaging tests label Jan 17, 2025
@cosmo0920 cosmo0920 marked this pull request as ready for review January 20, 2025 08:12
@cosmo0920 cosmo0920 added this to the Fluent Bit v4.0.0 milestone Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-required ok-package-test Run PR packaging tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Systemd service race causes incorrect hostname on Amazon Linux 2
2 participants