[Auditbeat] Add user metricset

[cwurm]

Adds a list of users with all their information to the host metricset. Works on Linux and macOS by using getpwent(3).

[ruflin]

My suggestion would be to add a separate metricset for host_user which sends an event for each user.

@andrewkroh FYI

[cwurm]

Thanks, @andrewkroh and @ruflin. I've (hopefully) addressed all comments for now. This is now its own metricset, sending each user as its own document, and detecting added/removed/changed users.

I'm still working on adding persistence across Auditbeat restarts, and we have some outstanding naming discussions (I put event.type / event.type_detail for now, but those are TBD).

So I'm keeping the PR in in progress for now, and you don't need to do another review at this point.

[cwurm]

I think this is ready for another review.

The user metricset now does everything that it should do. It reads (via C functions) /etc/passwd, /etc/shadow, and /etc/group to collect all relevant information about a user.

It can detect new users, deleted users, changes to users (e.g. groups), and - as a special distinct category - password changes.

Periodically, it sends all user information as state (frequency controlled by the config values state.period and user.state.period - default 12h). Otherwise, it periodically (controlled by the usual period config - very low values are possible and recommended since when no changes have occurred it only triggers three stat syscalls) checks the ctime of the above files, reads them if the ctime has changed, detects changes compared to its internal cache, and reports changes.

The cache is persisted to disk in a beat.db file (already used by the file_integrity module) after every Fetch and on Close. It contains all current user information incl. a 10-character excerpt of the password hash from /etc/shadow (necessary to detect any password changes incl. a direct edit of /etc/shadow - however, that excerpt is not output to ES or any other output).

Possible future enhancements:

• Use inotify instead of periodic stat calls to detect possible changes.
• Read /etc/sudoers to add sudo information to users.

[webmat]

LGTM overall.

I notice the metricset still needs to be changed to system_audit, but other than that, looking good. I've left a few comments here and there.

Other major thing is about using user.id to save the uid (see comment about this for the full discussion)

[andrewkroh]

Overall this LGTM. I left a few minor comments on the latest updates.

[cwurm]

After a conversation with @andrewkroh I changed the password change detection to use a SHA-512 hash of the full encrypted password (rather than the last 10 characters) - but excluding the salt. This should make sure that we detect any possible password change.

I merged master into the feature branch and rebased this PR, so the commits are now displayed all at the end. There are two commits since the last review: Switch to new UUID library. and
Use full encrypted password for change detection, store as SHA-512 hash. .

@andrewkroh let me know if this still looks good so I can merge.