Skip to content

feat: Scripted input support requirement test #442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 11, 2021
Merged

feat: Scripted input support requirement test #442

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ed3ee93
feat: Scripted input support requirement test
nandinivij Nov 2, 2021
b76294e
added transport type file monitor
nandinivij Nov 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions pytest_splunk_addon/standard_lib/event_ingestors/requirement_event_ingester.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,25 @@ def get_events(self):
LOGGER.info(
f"sending data transport_type:forwarder/uf_file_monitor filename:{filename} "
)
elif transport_type in (
"scripted_input",
"scripted input",
Copy link
Contributor

@nariyanibhargav nariyanibhargav Nov 10, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have "scripted input" and "scripted_input"?
How are transport type values decided?

Copy link
Contributor Author

@nandinivij nandinivij Nov 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In requirement files we specify transport types- there are these two common variations in existing logs.
Transport type values are added by the dev team to requirement logs based on the ingestion type of the log in the addon

Copy link

@ghost ghost Nov 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we accept only 1 one of them?
Not to create options where we do not need to.

Copy link
Contributor Author

@nandinivij nandinivij Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we will document only 1 version so that we don't use others in the future, but we are allowing multiple versions to avoid going back and changing existing log files. It is the case with modinput too

Copy link
Contributor

@nariyanibhargav nariyanibhargav Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nandinivij Can we ensure we document official usage for input types which is sync with PSA input types?

Copy link
Contributor

@nariyanibhargav nariyanibhargav Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also just for a record are there too many addons which use variations if not we can have them to use same version and ensure consistency and remove unwanted support from here? Do we have a list?

Copy link
Contributor Author

@nandinivij nandinivij Nov 11, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bhargav-nariyani-crest I understand the concern, but this is beyond the scope of this PR. The logs exist before we supported them to work with the requirement test. These are lexical variations and this is to facilitate requirement test work without changing existing logs.
This does not change the transport type we are using in PSA it is the defined transport type in the requirement log.

pytest-splunk-addon/tests/requirement_test_scripted/sample_requirement_test_scripted.log

Line 11 in b76294e

<transport type="scripted input" host="sample_host" source="Unix:Service" sourcetype ="Unix:Service" />

Created a Jira to eliminate various variations in transport type from TA'sADDON-44372

Copy link
Contributor

@nariyanibhargav nariyanibhargav Nov 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nandinivij .

"hec_raw",
):
transport_type = "scripted_input"
host, source, sourcetype = self.extract_params(
event_tag
)
LOGGER.info(
f"sending data transport_type:scripted_input or hec_raw filename:{filename} "
)
elif transport_type == "file_monitor":
host, source, sourcetype = self.extract_params(
event_tag
)
LOGGER.info(
f"sending data transport_type:file_monitor filename:{filename} "
)
else:
transport_type = "default"
unescaped_event = self.extract_raw_events(event_tag)
Expand Down
17 changes: 6 additions & 11 deletions pytest_splunk_addon/standard_lib/requirement_tests/test_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,11 @@ def generate_cim_req_params(self):
"dbx",
"windows_input",
"hec_event",
"scripted_input",
"scripted input",
"hec_raw",
"file_monitor",
"forwarder",
):
host, source, sourcetype = self.extract_params(event_tag)
host, source, sourcetype = self.escape_host_src_srctype(
Expand All @@ -160,18 +165,8 @@ def generate_cim_req_params(self):
"source": source,
"sourcetype": sourcetype,
}
elif transport_type.lower() == "forwarder":
host, source, sourcetype = self.extract_params(event_tag)
host, source, sourcetype = self.escape_host_src_srctype(
host, source, sourcetype
)
transport_type_params = {
"host": host,
"source": source,
"sourcetype": sourcetype,
}
else:
# todo: non syslog/modinput events are skipped currently until we support it
# todo: non syslog/modinput/non forwarder/scripted_input events are skipped currently until we support it
continue

escaped_event = self.escape_char_event(unescaped_event)
Expand Down
9 changes: 9 additions & 0 deletions tests/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -770,3 +770,12 @@
TA_REQUIREMENTS_UF_FAILED = [
"*test_splunk_app_requirements_uf.py::Test_App::test_requirement_params* FAILED*",
]

TA_REQUIREMENTS_SCRIPTED_PASSED = [
"*test_splunk_app_requirements_scripted.py::Test_App::test_events_with_untokenised_values* PASSED*",
"*test_splunk_app_requirements_scripted.py::Test_App::test_requirement_params* PASSED*",
]

TA_REQUIREMENTS_SCRIPTED_FAILED = [
"*test_splunk_app_requirements_scripted.py::Test_App::test_requirement_params* FAILED*",
]
37 changes: 37 additions & 0 deletions tests/requirement_test_scripted/sample_requirement_test_scripted.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
<?xml version="1.0"?>
<device>
<vendor>Nix</vendor>
<product>Nix</product>
<version id="6,7" os="CentOS"/>
<version id="6.9,7.4,8.0" os="RHEL"/>
<version id="16.04" os="Ubuntu"/>
<version id="10,11,11.3" os="Solaris"/>
<version id="10.11,10.12" os="MAC OS X"/>
<event code="" name="service" format="">
<transport type="scripted input" host="sample_host" source="Unix:Service" sourcetype ="Unix:Service" />
<source>
<comment>Unix service information. Data is collected from live instance. Data is generated from systemctl command. </comment>
</source>
<raw>
<![CDATA[Tue May 26 01:38:02 PDT 2020 type=systemctl UNIT=whoopsie.service, LOADED=not-found, ACTIVE=inactive, SUB=dead, DESCRIPTION="whoopsie.service"]]>
</raw>
<cim>
<models>
<model>Endpoint:Services</model>
</models>
<cim_fields>
<field name="service" value="whoopsie.service"/>
<field name="service_name" value="whoopsie.service"/>
<field name="status" value="stopped"/>
<field name="start_mode" value="Auto"/>
<field name="vendor_product" value="nix"/>
</cim_fields>
<missing_recommended_fields>
<field>service_id</field>
<field>service_path</field>
<field>user</field>
</missing_recommended_fields>
</cim>
<test></test>
</event>
</device>
38 changes: 38 additions & 0 deletions tests/requirement_test_scripted/sample_requirement_test_scripted_failure.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
<?xml version="1.0"?>
<device>
<vendor>Nix</vendor>
<product>Nix</product>
<version id="6,7" os="CentOS"/>
<version id="6.9,7.4,8.0" os="RHEL"/>
<version id="16.04" os="Ubuntu"/>
<version id="10,11,11.3" os="Solaris"/>
<version id="10.11,10.12" os="MAC OS X"/>
<event code="" name="service" format="">
<transport type="scripted input" host="sample_host" source="Unix:Service" sourcetype ="Unix:Service" />
<source>
<comment>Unix service information. Data is collected from live instance. Data is generated from systemctl command. </comment>
</source>
<raw>
<![CDATA[Tue May 26 01:38:02 PDT 2020 type=systemctl UNIT=whoopsie.service, LOADED=not-found, ACTIVE=inactive, SUB=dead, DESCRIPTION="whoopsie.service"]]>
</raw>
<cim>
<models>
<model>Endpoint:Services</model>
</models>
<cim_fields>
<field name="dest" value="qa-ubuntu-105"/>
<field name="service" value="whoopsie.service"/>
<field name="service_name" value="whoopsie.service"/>
<field name="status" value="stopped"/>
<field name="start_mode" value="Auto"/>
<field name="vendor_product" value="nix"/>
</cim_fields>
<missing_recommended_fields>
<field>service_id</field>
<field>service_path</field>
<field>user</field>
</missing_recommended_fields>
</cim>
<test></test>
</event>
</device>
2 changes: 1 addition & 1 deletion tests/requirement_test_uf/sample_requirement_test_uf_failure.log
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<version id="13.21" />
<event code="" name="failed_login1,sshd_authentication1" format="syslog">
<version id="7.2"/>
<transport type="forwarder" host="sample_host" source="sample_source" sourcetype = "aix_secure" />
<transport type="forwarder" host="sample_host2" source="sample_source2" sourcetype = "aix_secure" />
<source>
<comment>Logs for Failed login attempt.(Live Sample)</comment>
</source>
Expand Down
53 changes: 53 additions & 0 deletions tests/test_splunk_addon.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,12 @@ def setup_test_dir(testdir):
),
os.path.join(testdir.tmpdir, "tests/requirement_test_uf"),
)
shutil.copytree(
os.path.join(
testdir.request.config.invocation_dir, "tests/requirement_test_scripted"
),
os.path.join(testdir.tmpdir, "tests/requirement_test_scripted"),
)

shutil.copy(
os.path.join(testdir.request.config.invocation_dir, "Dockerfile.splunk"),
Expand Down Expand Up @@ -640,3 +646,50 @@ def empty_method():

# make sure that that we get a non '0' exit code for the testsuite as it contains failure
assert result.ret != 0


@pytest.mark.docker
def test_splunk_app_requirements_scripted(testdir):
"""Make sure that pytest accepts our fixture."""

testdir.makepyfile(
"""
from pytest_splunk_addon.standard_lib.addon_basic import Basic
class Test_App(Basic):
def empty_method():
pass
"""
)

shutil.copytree(
os.path.join(testdir.request.fspath.dirname, "addons/TA_requirement_test_uf"),
os.path.join(testdir.tmpdir, "package"),
)

setup_test_dir(testdir)
SampleGenerator.clean_samples()
Rule.clean_rules()

# run pytest with the following cmd args
result = testdir.runpytest(
"--splunk-type=docker",
"-v",
"-m splunk_searchtime_requirements",
"--search-interval=4",
"--search-retry=4",
"--search-index=*,_internal",
"--requirement-test=tests/requirement_test_scripted",
)
logger.info(result.outlines)
logger.info(len(constants.TA_REQUIREMENTS_SCRIPTED_PASSED))
logger.info(len(constants.TA_REQUIREMENTS_SCRIPTED_FAILED))
result.stdout.fnmatch_lines_random(
constants.TA_REQUIREMENTS_SCRIPTED_PASSED
+ constants.TA_REQUIREMENTS_SCRIPTED_FAILED
)
result.assert_outcomes(
passed=len(constants.TA_REQUIREMENTS_SCRIPTED_PASSED), failed=1
)

# make sure that that we get a non '0' exit code for the testsuite as it contains failure
assert result.ret != 0