Skip to content

Commit

Permalink
Merge pull request #66 from fluent/support-remoting-access-parameters
Browse files Browse the repository at this point in the history 
Support remoting access parameters
  • Loading branch information
@cosmo0920
cosmo0920 authored Sep 16, 2020
2 parents 27cd315 + c41584a commit d9ab94d
Show file tree
Hide file tree
Showing 4 changed files with 226 additions and 21 deletions.
37 changes: 37 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging
# preserve_qualifiers_on_hash true # default is false.
# read_all_channels false # default is false.
# description_locale en_US # default is nil. It means that system locale is used for obtaining description.
# refresh_subscription_interval 10m # default is nil. It specifies refresh interval for channel subscriptions.
<storage>
@type local # @type local is the default.
persistent true # default is true. Set to false to use in-memory storage.
Expand All @@ -161,6 +162,10 @@ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging
# <subscribe>
# channles application, system
# read_existing_events false # read_existing_events should be applied each of subscribe directive(s)
# remote_server 127.0.0.1 # Remote server ip/fqdn
# remote_domain WORKGROUP # Domain name
# remote_username fluentd # Remoting access account name
# remote_password changeme! # Remoting access account password
# </subscribe>
</source>

Expand Down Expand Up @@ -190,6 +195,7 @@ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging
|`preserve_qualifiers_on_hash` | (option) When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys. When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers". Default is `false`.|
|`read_all_channels`| (option) Read from all channels. Default is `false`|
|`description_locale`| (option) Specify description locale. Default is `nil`. See also: [Supported locales](https://github.com/fluent-plugins-nursery/winevt_c#multilingual-description) |
|`refresh_subscription_interval`|(option) It specifies refresh interval for channel subscriptions. Default is `nil`.|
|`<subscribe>` | Setting for subscribe channels. |

##### subscribe section
Expand All @@ -198,6 +204,10 @@ fluentd Input plugin for the Windows Event Log using newer Windows Event Logging
|:----- |:----- |
|`channels` | One or more of {'application', 'system', 'setup', 'security'}. If you want to read 'setup' or 'security' logs, you must launch fluentd with administrator privileges. |
|`read_existing_events` | (option) Read the entries which already exist before fluentd is started. Defaults to `false`. |
|`remote_server` | (option) Remoting access server ip address/fqdn. Defaults to `nil`. |
|`remote_domain` | (option) Remoting access server joining domain name. Defaults to `nil`. |
|`remote_username` | (option) Remoting access access account's username. Defaults to `nil`. |
|`remote_password` | (option) Remoting access access account's password. Defaults to `nil`. |


**Motivation:** subscribe directive is designed for applying `read_existing_events` each of channels which is specified in subscribe section(s).
Expand Down Expand Up @@ -234,6 +244,33 @@ This configuration can be handled as:
* "Application" and "Security" channels just tailing
* "HardwareEvent" channel read existing events before launching Fluentd

###### Remoting access

`<subscribe>` section supports remoting access parameters:

* `remote_server`
* `remote_domain`
* `remote_username`
* `remote_password`

These parameters are only in `<subscribe>` directive.

Note that before using this feature, remoting access users should belong to "Event Log Readers" group:

```console
> net localgroup "Event Log Readers" <domain\username> /add
```

And then, users also should set up their remote box's Firewall configuration:

```console
> netsh advfirewall firewall set rule group="Remote Event Log Management" new enable=yes
```

As a security best practices, remoting access account _should not be administrator account_.

For graphical instructions, please refer to [Preconfigure a Machine to Collect Remote Windows Events | Sumo Logic](https://help.sumologic.com/03Send-Data/Sources/01Sources-for-Installed-Collectors/Remote-Windows-Event-Log-Source/Preconfigure-a-Machine-to-Collect-Remote-Windows-Events) document for example.

##### Available keys

This plugin reads the following fields from Windows Event Log entries. Use the `keys` configuration option to select a subset. No other customization is allowed for now.
Expand Down
4 changes: 2 additions & 2 deletions fluent-plugin-winevtlog.gemspec
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Gem::Specification.new do |spec|
spec.add_development_dependency "bundler"
spec.add_development_dependency "rake"
spec.add_development_dependency "test-unit", "~> 3.2.0"
spec.add_development_dependency "nokogiri", [">= 1.11.pre", "< 1.12"]
spec.add_development_dependency "nokogiri", [">= 1.10", "< 1.12"]
spec.add_development_dependency "fluent-plugin-parser-winevt_xml", ">= 0.1.2"
spec.add_runtime_dependency "fluentd", [">= 0.14.12", "< 2"]
spec.add_runtime_dependency "win32-eventlog"
spec.add_runtime_dependency "winevt_c", ">= 0.8.1"
spec.add_runtime_dependency "winevt_c", ">= 0.9.1"
end
116 changes: 101 additions & 15 deletions lib/fluent/plugin/in_windows_eventlog2.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ module Fluent::Plugin
class WindowsEventLog2Input < Input
Fluent::Plugin.register_input('windows_eventlog2', self)

class ReconnectError < Fluent::UnrecoverableError; end

helpers :timer, :storage, :parser

DEFAULT_STORAGE_TYPE = 'local'
Expand Down Expand Up @@ -43,10 +45,15 @@ class WindowsEventLog2Input < Input
config_param :preserve_qualifiers_on_hash, :bool, default: false
config_param :read_all_channels, :bool, default: false
config_param :description_locale, :string, default: nil
config_param :refresh_subscription_interval, :time, default: nil

config_section :subscribe, param_name: :subscribe_configs, required: false, multi: true do
config_param :channels, :array
config_param :read_existing_events, :bool, default: false
config_param :remote_server, :string, default: nil
config_param :remote_domain, :string, default: nil
config_param :remote_username, :string, default: nil
config_param :remote_password, :string, default: nil, secret: true
end

config_section :storage do
Expand All @@ -68,9 +75,12 @@ def initalize

def configure(conf)
super
@session = nil
@chs = []
@subscriptions = {}
@all_chs = Winevt::EventLog::Channel.new
@all_chs.force_enumerate = false
@timers = {}

if @read_all_channels
@all_chs.each do |ch|
Expand All @@ -81,14 +91,22 @@ def configure(conf)

@read_existing_events = @read_from_head || @read_existing_events
if @channels.empty? && @subscribe_configs.empty? && !@read_all_channels
@chs.push(['application', @read_existing_events])
@chs.push(['application', @read_existing_events, nil])
else
@channels.map {|ch| ch.strip.downcase }.uniq.each do |uch|
@chs.push([uch, @read_existing_events])
@chs.push([uch, @read_existing_events, nil])
end
@subscribe_configs.each do |subscribe|
if subscribe.remote_server
@session = Winevt::EventLog::Session.new(subscribe.remote_server,
subscribe.remote_domain,
subscribe.remote_username,
subscribe.remote_password)

log.debug("connect to remote box (server: #{subscribe.remote_server}) domain: #{subscribe.remote_domain} username: #{subscribe.remote_username})")
end
subscribe.channels.map {|ch| ch.strip.downcase }.uniq.each do |uch|
@chs.push([uch, subscribe.read_existing_events])
@chs.push([uch, subscribe.read_existing_events, @session])
end
end
end
Expand Down Expand Up @@ -137,12 +155,73 @@ def unsupported_locale?(locale, description_locale)
def start
super

@chs.each do |ch, read_existing_events|
subscribe_channel(ch, read_existing_events)
refresh_subscriptions
if @refresh_subscription_interval
timer_execute(:in_windows_eventlog_refresh_subscription_timer, @refresh_subscription_interval, &method(:refresh_subscriptions))
end
end

def subscribe_channel(ch, read_existing_events)
def shutdown
super

@subscriptions.keys.each do |ch|
subscription = @subscriptions.delete(ch)
if subscription
subscription.cancel
log.debug "channel (#{ch}) subscription is canceled."
end
end
end

def retry_on_error(channel, times: 15)
try = 0
begin
log.debug "Retry to subscribe for #{channel}...." if try > 1
try += 1
yield
log.info "Retry to subscribe for #{channel} succeeded." if try > 1
try = 0
rescue Winevt::EventLog::Subscribe::RemoteHandlerError => e
raise ReconnectError, "Retrying limit is exceeded." if try > times
log.warn "#{e.message}. Remaining retry count(s): #{times - try}"
sleep 2**try
retry
end
end

def refresh_subscriptions
clear_subscritpions

@chs.each do |ch, read_existing_events, session|
retry_on_error(ch) do
ch, subscribe = subscription(ch, read_existing_events, session)
@subscriptions[ch] = subscribe
end
end
subscribe_channels(@subscriptions)
end

def clear_subscritpions
@subscriptions.keys.each do |ch|
subscription = @subscriptions.delete(ch)
if subscription
if subscription.cancel
log.debug "channel (#{ch}) subscription is cancelled."
subscription.close
log.debug "channel (#{ch}) subscription handles are closed forcibly."
end
end
end
@timers.keys.each do |ch|
timer = @timers.delete(ch)
if timer
event_loop_detach(timer)
log.debug "channel (#{ch}) subscription watcher is detached."
end
end
end

def subscription(ch, read_existing_events, remote_session)
bookmarkXml = @bookmarks_storage.get(ch) || ""
bookmark = nil
if bookmark_validator(bookmarkXml, ch)
Expand All @@ -151,7 +230,7 @@ def subscribe_channel(ch, read_existing_events)
subscribe = Winevt::EventLog::Subscribe.new
subscribe.read_existing_events = read_existing_events
begin
subscribe.subscribe(ch, "*", bookmark)
subscribe.subscribe(ch, "*", bookmark, remote_session)
if !@render_as_xml && @preserve_qualifiers_on_hash
subscribe.preserve_qualifiers = @preserve_qualifiers_on_hash
end
Expand All @@ -161,8 +240,15 @@ def subscribe_channel(ch, read_existing_events)
subscribe.render_as_xml = @render_as_xml
subscribe.rate_limit = @rate_limit
subscribe.locale = @description_locale if @description_locale
timer_execute("in_windows_eventlog_#{escape_channel(ch)}".to_sym, @read_interval) do
on_notify(ch, subscribe)
[ch, subscribe]
end

def subscribe_channels(subscriptions)
subscriptions.each do |ch, subscribe|
@timers[ch] = timer_execute("in_windows_eventlog_#{escape_channel(ch)}".to_sym, @read_interval) do
on_notify(ch, subscribe)
end
log.debug "channel (#{ch}) subscription is subscribed."
end
end

Expand Down Expand Up @@ -222,12 +308,12 @@ def on_notify_xml(ch, subscribe)
end
end
end
router.emit_stream(@tag, es)
@bookmarks_storage.put(ch, subscribe.bookmark)
rescue Winevt::EventLog::Query::Error => e
log.warn "Invalid XML data", error: e
log.warn "Invalid XML data on #{ch}.", error: e
log.warn_backtrace
end
router.emit_stream(@tag, es)
@bookmarks_storage.put(ch, subscribe.bookmark)
end

def on_notify_hash(ch, subscribe)
Expand All @@ -252,12 +338,12 @@ def on_notify_hash(ch, subscribe)
parse_desc(h) if @parse_description
es.add(Fluent::Engine.now, h)
end
router.emit_stream(@tag, es)
@bookmarks_storage.put(ch, subscribe.bookmark)
rescue Winevt::EventLog::Query::Error => e
log.warn "Invalid Hash data", error: e
log.warn "Invalid Hash data on #{ch}.", error: e
log.warn_backtrace
end
router.emit_stream(@tag, es)
@bookmarks_storage.put(ch, subscribe.bookmark)
end

#### These lines copied from in_windows_eventlog plugin:
Expand Down
Loading

0 comments on commit d9ab94d

Please sign in to comment.