Relationship between Detections and Atomic Red Team test cases (or other test case library)

[jbrianmoss]

Good morning, I am wondering if there is a mapping kept between Detections and an Atomic Red Team test case(s)? I was half expecting this to be a tag in the Detection's YAML file, but it appears the only reference is in the free text description of the Detection and not consistently for all Detections. I see that the Cyber Range supports Atomic Red Team so maybe this mapping might existing somewhere else?

If this mapping does not exist anywhere, the a follow-up question would be if there are any suggestions on how best to implement the mapping in a way that would be acceptable to the project's maintainers. We are currently thinking we would need to create this mapping, and if that is the case we would like to do it in such a way that it could help the community.

Kind Regards,

Brian

[jbrianmoss]

If this is better place as a question to Splunk Support, please let me know and we can go that route instead.

[josehelps]

@jbrianmoss We today do not have an "explicit" mapping to Atomic Red Team, but for sure there is an implicit one, and that is mainly drive my our mitre_attack_id tag seen here: https://github.com/splunk/security_content/search?q=mitre_attack_id+path%3Adetections%2F .. these usually map to the exact atomic technique to use. To further confirm this you can look into the tests files, usually they refernce the attack dataset source to be atomic red team. For example: https://github.com/splunk/security_content/blob/develop/tests/endpoint/attempt_to_add_certificate_to_untrusted_store.test.yml#L10. With that said we have also considered including an array of atomic guids to match up with each test file as documented in our PR templates: https://github.com/splunk/security_content/blob/develop/.github/pull_request_template.md?plain=1#L11. We have not officially implemented this yet and or enforced it with the team but this will introduce a "explicit" relationship with Atomic Red Team. Let me know if this helps 😄

[jbrianmoss]

Thanks @d1vious. I don't completely follow what you mean regarding "considered including an array of atomic guids to match up with each test file as documented in our PR templates", but I will elaborate my question.

What does Splunk Threat Research use internally to track MITRE Procedure coverage? Do you use Atomic Red Team's Atomic Test #'s, or PurpleSharp's Variation #'s, or something else?

What we are really looking for is to track our detection and tests at the "Procedure" level. This Vectr documentation explains it better than I can https://docs.vectr.io/user/best-practices/

For VECTR to track results of Purple Team activities over time, some way is needed to uniquely identify a specific test procedure. The MITRE Technique or Sub-Technique ID is not specific enough to accomplish this. For example, there may be many unique test procedures for T1047 - Windows Management Instrumentation.

It's important to create individual tests confirming detection and response for each of the different attack scenarios that may be categorized by a specific ATT&CK Technique. To accomplish this, VECTR's Test Cases map to the MITRE Enterprise ATT&CK "Procedure" concept, and VECTR uses Test Case Templates to uniquely identify a test procedure. These templates can be managed in the Administration section of VECTR, as described below.

If using Atomic Red Team's Atomic Tests is what you use internally to represent MITRE Procedures, what I was imagining was in the detection YAML file referencing the Atomic Red Team's Atomic Test GUID(s) that are covered by the detection. For example https://github.com/splunk/security_content/blob/develop/detections/endpoint/attempt_to_add_certificate_to_untrusted_store.yml#L25 references https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1553.004/T1553.004.md, but I was hoping at a minimum it would instead reference https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1553.004/T1553.004.md#atomic-test-4---install-root-ca-on-windows and https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1553.004/T1553.004.md#atomic-test-5---install-root-ca-on-windows-with-certutil, because the detection only supports Splunk_TA_microsoft_sysmon, assuming it covers both of these Procedures. Looking at the test data you referenced, https://github.com/splunk/attack_data/blob/master/datasets/attack_techniques/T1553.004/atomic_red_team/atomic_red_team.yml#L5, it appears only https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1553.004/T1553.004.md#atomic-test-5---install-root-ca-on-windows-with-certutil is covered by the detection, or at least the test data only test for T1553.004-5.

What I was really hoping for was to add additional values to the "mitre_attack_id" key introduce a new key such as "atomic_test" with value(s) of either the atomic tests' automatically generated GUID(s), or the link to the specific test(s) in the .md file.

For the example detection https://github.com/splunk/security_content/blob/develop/detections/endpoint/attempt_to_add_certificate_to_untrusted_store.yml add in:

atomic_test:

• https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1553.004/T1553.004.md#atomic-test-5---install-root-ca-on-windows-with-certutil (or)
• 5fdb1a7a-a93c-4fbe-aa29-ddd9ef94ed1f

Thanks in advance for the help and explanations.

Brian