Provide APIs to manage pipeline

[suyograo]

Currently, whenever Logstash configuration needs to be modified, you have to edit a configuration file and restart the Logstash process. This process requires the user to either temporarily halt the pipeline or accept the fact that an interruption in processing will occur. While file-based configuration works well (especially with Puppet/Chef), it becomes increasing difficult to deploy Logstash instances. We plan to add a REST based Logstash management API to make it easier to interact with Logstash. This will allow dynamic configuration updates without the need to restart the process. To be clear, at this time we are not looking to change the existing configuration syntax. Also, file-based configuration will continue to be supported.

Important manageability functions for this end-point are:

• reload - restarts a paused pipeline instance with new config and instantiates all the plugins
• add - Add specific parts of the pipeline dynamically. For example, we may want to add a file input to follow a new file.
• destroy - sends shutdown signal to all plugins and eventually stops
• status - include pipeline state, list of plugins, the config itself
• pause - pauses the pipeline
• unpause - resumes the paused pipeline

When interacting with the manageability APIs, care should be taken to not drop in-flight messages.