Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions detection_rules/etc/test_toml.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,14 @@
]
}
},
{
"metadata": {
"field": "value"
},
"rule": {
"path": "?:\\\\Windows\\\\Sys?????\\\\x5lrs.dll"
}
},
{
"metadata": {
"field": "value"
Expand Down
3 changes: 3 additions & 0 deletions detection_rules/rule_formatter.py
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,9 @@ def dump_str(self, v: str | NonformattedField) -> str:
return "\n".join([TRIPLE_SQ] + [self._old_dump_str(line)[1:-1] for line in lines] + [TRIPLE_SQ])
if raw:
return f"'{lines[0]:s}'"
# In the toml library there is a magic replace for \\\\x -> u00 that we wish to avoid until #4979 is resolved
if "\\\\x" in v:
return f'"{v!s}"'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will want a new unit test in test_toml_formatter.py

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ agreed, added new unit test that looks for this specifically. It also will fail if the path behavior changes. E.g. currently we expect \\\\ to be formatted to \\. If this changes the unit test will fail on purpose as we want to match Query DSL's path handling which does this as well.

return self._old_dump_str(v)

def _dump_flat_list(self, v: Iterable[Any]) -> str:
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "detection_rules"
version = "1.3.23"
version = "1.3.24"
description = "Detection Rules is the home for rules used by Elastic Security. This repository is used for the development, maintenance, testing, validation, and release of rules for Elastic Security’s Detection Engine."
readme = "README.md"
requires-python = ">=3.12"
Expand Down
10 changes: 9 additions & 1 deletion tests/test_toml_formatter.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,12 @@ def test_formatter_rule(self):

def test_formatter_deep(self):
"""Test that the data remains unchanged from formatting."""
self.compare_test_data(self.test_data[1:])
self.compare_test_data(self.test_data[2:])

def test_formatter_paths(self):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add another test to make sure \\\\x is not converted to \\u00?

@eric-forte-elastic eric-forte-elastic Aug 18, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this test not accomplish that?

    def test_formatter_paths(self):
        """Test that paths are handled as expected with toml lib."""
        with self.assertRaisesRegex(
            AssertionError,
            r'\+ {"metadata": {"field": "value"}, "rule": {"path": "\?:\\\\Windows\\\\Sys\?\?\?\?\?\\\\x5lrs\.dll"}}',
        ):
            self.compare_test_data([self.test_data[1]])

This output checks the need for "?:\\\\Windows\\\\Sys?????\\\\x5lrs.dll" to explicitly become "?:\\Windows\\Sys?????\\x5lrs.dll"

If the \\\\x is converted to \\u00 it will fail. It will fail if \\\\x is not converted to exactly \\x (e.g. \\\\x -> \\\\x will also fail as is its intention)

The regex filter to catch the specific transformation

image

The + line needs to match exactly in order for this unit test to pass.

Catches the \\u00 in its current form

image

Is this accomplishing what you are looking for?

"""Test that paths are handled as expected in with toml lib."""
with self.assertRaisesRegex(
AssertionError,
r'\+ {"metadata": {"field": "value"}, "rule": {"path": "\?:\\\\Windows\\\\Sys\?\?\?\?\?\\\\x5lrs\.dll"}}',
):
self.compare_test_data([self.test_data[1]])
Loading