DOCUMENTATION: Improve documentation for configuring Playbook alerts and custom filters

[thedeadliestcatch]

I'm reproducing the original description prior to its conversion into a discussion (#12705) which is unfitting and dismissive, considering that it pertains to 1) a clear issue that can be addressed and tracked as such 2) a general user-base issue/improvement

The documentation for configuring Playbook -alert- filters is sorely lacking.

For instance, valuable documentation missing includes:

Compound filters depending on multiple conditionals (ex. host, process executable, parent process executable).
Compound filters using hosts lists.
How to use any context variables in the filter.

There are many playbook rules that remain inactive by default that are quite useful in real-world setups, but need fine tuning due to false positives, for example in container environments. The documentation currently has no real content about configuration of said filters, and the publicly available information is also deficient. This most likely results in people just deactivating the problematic rules.... a zero-sum game.

Please do not frivolously convert issues into "discussions" as it makes them impossible to track, and more often than not, they are never seen again. It also inflates the apparent number of closed issues versus open ones, which is not a good measure of effective QA, even if it makes it more satisfying for the team to wake up to an (artificially) short list of problems.

Case in point, the present situation with #12653 originated from several (not just one) messages after the original issue was dismissed here. I'm sure we can do better.

[thedeadliestcatch]

A follow-up:

The current documentation is limited to https://docs.securityonion.net/en/2.3/playbook.html#tuning-plays

The sofilter syntax is important - add as many top-level filter clauses as you need, but they should all start with sofilter - for example sofilter1, sofilter2

The examples should include multiple cases, the existent one is also potentially meaningless to most users, since we are not explaining where User comes from:

sofilter:
  User: SA_ITOPS

There should be proper clarification to what is permitted or supported, where to find the key names (users, process name, etc), and if these can be applied in a conditional way, etc.

Amusingly enough, this has become the first search engine hit for documentation on filtering Playbook alerts :-)

[thedeadliestcatch]

Let's take the following rule/play:

https://github.com/SigmaHQ/sigma/blob/master/rules/linux/file_event/file_event_lnx_wget_download_file_in_tmp_dir.yml

Tentatively, an user might find that a legitimate script generates a false positive while downloading to the /tmp directory (say, in a LXC container), and the existent documentation will provide no useful information as to how he can adapt the play/rule to his environment.

Given the following:

detection:
    selection:
        Image|endswith: '/wget'
        TargetFilename|startswith:
            - '/tmp/'
            - '/var/tmp/'
    condition: selection

The documentation ideally should answer:

• How can the user configure a conditional mutual exclusion case where TargetFilename and Image match but another context variable does not (ex. Match against a parent process image, or the running user, or the host itself).
• What context variables are available and how he can use them to produce a "discard" path for the alert without an overly inclusive set of conditionals.

[TOoSmOotH]

There was only a single discussion I could find about Suricata and pfsense and that was the one I responded to originally that we are looking into it further. #12653 did come from that discussion.

I will skip over the part where you were @ me on a holiday weekend here in the US for updates.

To address the above "issue". We are about to launch a new feature called "Detections" in 2.4.70 which will require us to overhaul the existing documentation on this subject. You put your issue in during a holiday weekend here in the US. I was not going to drop everything and explain this at that time as I was enjoying time with my family. Your issue is a duplicate to one we are tracking internally which is another reason it was converted to a discussion. You then put this in as an issue with your hot take on our process before I had time to respond to the original one before our workday here in the US started.

Our workflow is everything starts as a discussion that comes from the community. As you can imagine from a lot of the discussions here, we use this to filter what the actual issue could be and then create an issue based off of that discussion. We link to the issue from the discussion typically for bugs. We do sometimes accept issues from the community when they are including a PR to fix the issue.

We provide community support for free with a best effort approach. We do this because we feel the community's ideas and issues are critical to improving our product.

[thedeadliestcatch]

There was only a single discussion I could find about Suricata and pfsense and that was the one I responded to originally that we are looking into it further. https://github.com/Security-Onion-Solutions/securityonion/issues/12653 did come from that discussion.

Without going further into it, as it is unrelated, if you look for messages discussing syslog and pfsense/consumption of Suricata EVE, you will find several of the same nature. I believe they go back beyond one major revision (older than 2.3.0). Do note that I make a distinction between a discussion or support request (which can come with poor writing style) and something like this current issue, which comes with generic, non-case specific details and it is a legitimate issue to track. But those discussions hinted at the issue: there is a missing pipeline or ingestion step.

I fully agree that it can happen and it does happen that users flood issue trackers with support or case-specific requests. There is no case against the need to convert those to discussions. But in this particular case, you pulled the trigger too fast too soon :-)

I will skip over the part where you were @ me on a holiday weekend here in the US for updates.

I'm effectively a customer paying with free QA and SO is an Open Source project (this is, in the end, how most pseudo commercial FOSS projects operate: you trade source availability for access to extensive QA resources for free), there is no demand or expectation implied for responding on out of office hours, but the mere mention of your nickname does not equate to someone ringing your phone in the middle of the weekend to do anything. I'm just making sure Github has a notification waiting for you when you come online at a time of your choosing.
I also do not know your role or when you converted the other issue into a discussion, but that is all irrelevant to us: I just want this issue fixed for your users.

Now back to the technical side: feel free to @ the person responsible for the forthcoming Detections changes. Do you have links to source or repos that I could look into and perhaps come back with suggestions for the documentation? I want to make sure I am suggesting something that will be practically possible with the new changes (ex. if the UI in Playbooks changes, the suggestions I made earlier might not be possible as-is). I'm happy to check them out and provide some help.

I also have a separate issue related to Suricata upstream rules missing (their 'stream' and reassembly rules are all missing in an up-to-date test setup I checked recently). I will check again after the last batch of updates.

[dougburks]

@thedeadliestcatch As Mike mentioned above, we have massive changes coming in the next version that will impact Playbook and its documentation. Because of that, we are quite busy and have neither the time nor the energy to continue to go back and forth like this. Please be patient and wait for the next version. There is no need to create any other issues on this topic.

Please review #1720. Per those guidelines, if you can't be patient, courteous, and respectful, then your account will be banned from this repo.

Thanks in advance for your understanding and patience.

[thedeadliestcatch]

@dougburks What repositories should we keep an eye on to observe the progress for the coming changes?

Please review #1720. Per those guidelines, if you can't be patient, courteous, and respectful, then your account will be banned from this repo.

I do not see where I have not been courteous and respectful, as for patience I do not see where I have been impatient either (subjective as that might be). A latent threat of a ban seems out of line, as is misrepresenting my behavior. If that happens in absence of objective reasons and is used frivolously, I (and my employer) reserve the right to approach the issue as publicly and diligently as we see fit. You are a FOSS project, you should be able to handle criticism without threatening your users. I also would like to make it obvious that we are interacting in writing, so I would appreciate it if my criticism did not result in gratuitous threats or misrepresentation of my person (that could result in reputational damage that I certainly do not deserve).

Thanks.

[dougburks]

@thedeadliestcatch A patient, courteous, and respectful reply would have been something like:

I understand that you're busy. I will respect your request to be patient and wait for the next release.

So I'll state very clearly one more time: we are quite busy and have neither the time nor the energy to continue to go back and forth like this.

Please respect our request to be patient and wait for the next version.

Let us do our jobs so that we can make the next version even better so that our entire defender community can better defend themselves against our actual adversaries.

There is no need to create any other issues or discussions on this topic.

Thanks.