Add ability to override configuration settings using environment variables.

[mingfang]

I'm trying to deploy the packetbeat agent to a cluster of machines using kubernetes.
However the ElasticSearch hostname needs to be configurable at deployment time.
I would like the ability to set that using environment variables,
e.g.

export PACKETBEAT_ELASTICSEARCH_HOST=dynamic-host

[tsg]

Good suggestion, we'll add it.

[albanm]

+1

How about using viper for a more complete config system ?

If you lack the time but are ok with the idea I am willing to attempt a pull request (total go newbie here, but this should not be too difficult).

[yoanisgil]

I think this an excellent idea. I have a config file which looks like this:

filebeat:
  prospectors:
    -
      paths:
        - "/var/log/supervisor/nginx*.log"
      input_type: log
      fields:
        source: nginx
        level: info
        environment: development
output:
    console:
        pretty: true
    logstash:
        hosts: ["logstash.mpl.ninja:5044"]

and I would like to be able to configure the list of logstash hosts as well as the environment setting. The reason I need to do so is because I need to provide support for dev/staging/production environments the later 2 being AWS Beanstalk environments which are updated via Jenkins.

Anyone working on this? I could give a try if this is something you're willing to implement.

[ruflin]

@yoanisgil As far as I know there is currently no one working on this. I somehow see the need for this especially for larger deployment but at the same time I'm worried about having one more unknown to debug, in case of issue. Right now we know exactly what the config file is.

@albanm I had a quick look at viper. Looks interesting, not sure if it is overkill for what we need.

Also have a look at #464

[mingfang]

Until there is an official solution, this is my current hack https://github.com/mingfang/docker-packetbeat-agent/blob/master/sv/packetbeat/run#L8. It's nothing innovative; just a simple sed hack.
For a nicer solution I would use confd instead.

[yoanisgil]

@mingfang same on this side. I had to rely on sed as a work around.

@ruflin I guess we should wait and see if this feature request raises more interest from the community.

[ruflin]

@mingfang I actually quite like this "hack" as it is simple and gets the job done.
@yoanisgil Agree

[andrewkroh]

This feature was added. Please see #715 for details.

[yoanisgil]

Thanks guys for adding this feature!