Update expected log files of Filebeat modules#25271

Closed

kvch wants to merge 1 commit intoelastic:7.13from

kvch:fix-filebeat-update-module-expected-logs

Contributor

kvch commented Apr 23, 2021

No description provided.


          Update expected log files

5a49a0e

botelastic Bot added the needs_team label

botelastic Bot commented Apr 23, 2021

This pull request doesn't have a Team:<team> label.

Contributor Author

kvch commented Apr 23, 2021

Closing in favour of #25272

kvch closed this

Contributor

elasticmachine commented Apr 23, 2021

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #25271 opened
Start Time: 2021-04-23T12:06:52.659+0000
Duration: 61 min 44 sec
Commit: 5a49a0e

Test stats 🧪

Test	Results
Failed	150
Passed	13523
Skipped	2299
Total	15972

Trends 🧪

Test errors

Expand to view the tests failures

> Show only the first 10 test failures

`Build&Test / x-pack/filebeat-build / test_fileset_file_001_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'saml', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'google_workspace.saml.initiated_by': 'idp', 'google_workspace.saml.status_code': 'SUCCESS_URI', 'google_workspace.saml.application_name': 'app', 'google_workspace.saml.second_level_status_code': 'SUCCESS_URI', 'google_workspace.saml.failure_type': 'failure_app_not_configured_for_user', 'google_workspace.saml.orgunit_path': 'ounit', 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"saml","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"login","name":"login_failure","parameters":[{"name":"application_name","value":"app"},{"name":"failure_type","value":"failure_app_not_configured_for_user"},{"name":"initiated_by","value":"idp"},{"name":"orgunit_path","value":"ounit"},{"name":"saml_second_level_status_code","value":"SUCCESS_URI"},{"name":"saml_status_code","value":"SUCCESS_URI"}]}}', 'event.provider': 'saml', 'event.module': 'gsuite', 'event.action': 'login_failure', 'event.id': '1', 'event.type': ['start'], 'event.category': ['authentication', 'session'], 'event.dataset': 'gsuite.saml', 'event.outcome': 'failure', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.event.type': 'login'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_001_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_001_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/saml/test/gsuite-saml-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '41e8efbc-fac2-4392-bbe3-2e41581761c2', 'hostnam...{'action': 'login_success', 'category': ['authentication', 'session'], 'dataset': 'gsuite.saml', 'id': '1', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'saml', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'google_workspace.saml.initiated_by': 'idp', 'google_workspace.saml.status_code': 'SUCCESS_URI', 'google_workspace.saml.application_name': 'app', 'google_workspace.saml.second_level_status_code': 'SUCCESS_URI', 'google_workspace.saml.failure_type': 'failure_app_not_configured_for_user', 'google_workspace.saml.orgunit_path': 'ounit', 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"saml","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"login","name":"login_failure","parameters":[{"name":"application_name","value":"app"},{"name":"failure_type","value":"failure_app_not_configured_for_user"},{"name":"initiated_by","value":"idp"},{"name":"orgunit_path","value":"ounit"},{"name":"saml_second_level_status_code","value":"SUCCESS_URI"},{"name":"saml_status_code","value":"SUCCESS_URI"}]}}', 'event.provider': 'saml', 'event.module': 'gsuite', 'event.action': 'login_failure', 'event.id': '1', 'event.type': ['start'], 'event.category': ['authentication', 'session'], 'event.dataset': 'gsuite.saml', 'event.outcome': 'failure', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.event.type': 'login'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_002_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'groups', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"groups","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"acl_change","name":"change_acl_permission","parameters":[{"name":"acl_permission","value":"can_add_members"},{"name":"group_email","value":"group@example.com"},{"name":"new_value_repeated","multiValue":["managers","members"]},{"name":"old_value_repeated","multiValue":["managers"]}]}}', 'event.provider': 'groups', 'event.module': 'gsuite', 'event.action': 'change_acl_permission', 'event.id': '1', 'event.type': ['group', 'change'], 'event.category': ['iam'], 'event.dataset': 'gsuite.groups', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'group.domain': 'example.com', 'group.name': 'group', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.groups.old_value': ['managers'], 'gsuite.groups.email': 'group@example.com', 'gsuite.groups.acl_permission': 'can_add_members', 'gsuite.groups.new_value': ['managers', 'members'], 'gsuite.event.type': 'acl_change'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_002_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_002_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/groups/test/gsuite-groups-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '35a9a354-2216-48ca-9413-766f848f44bb', 'hostnam...: 'change_basic_setting', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.groups', 'id': '1', ...}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'groups', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"groups","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"acl_change","name":"change_acl_permission","parameters":[{"name":"acl_permission","value":"can_add_members"},{"name":"group_email","value":"group@example.com"},{"name":"new_value_repeated","multiValue":["managers","members"]},{"name":"old_value_repeated","multiValue":["managers"]}]}}', 'event.provider': 'groups', 'event.module': 'gsuite', 'event.action': 'change_acl_permission', 'event.id': '1', 'event.type': ['group', 'change'], 'event.category': ['iam'], 'event.dataset': 'gsuite.groups', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'group.domain': 'example.com', 'group.name': 'group', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.groups.old_value': ['managers'], 'gsuite.groups.email': 'group@example.com', 'gsuite.groups.acl_permission': 'can_add_members', 'gsuite.groups.new_value': ['managers', 'members'], 'gsuite.event.type': 'acl_change'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_003_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"MOBILE_SETTINGS","name":"ACTION_CANCELLED","parameters":[{"name":"ACTION_ID","value":"id"},{"name":"ACTION_TYPE","value":"ACCOUNT_WIPE"},{"name":"DEVICE_ID","value":"id"},{"name":"DEVICE_TYPE","value":"type"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ACTION_CANCELLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['user', 'info'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.mobile.action.id': 'id', 'gsuite.admin.mobile.action.type': 'ACCOUNT_WIPE', 'gsuite.admin.device.id': 'id', 'gsuite.admin.device.type': 'type', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'MOBILE_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_003_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_003_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-mobile-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '87de6457-fdf5-4d59-af8e-7526a8d0a62a', 'hostnam...ent': {'action': 'COMPANY_DEVICE_DELETION', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"MOBILE_SETTINGS","name":"ACTION_CANCELLED","parameters":[{"name":"ACTION_ID","value":"id"},{"name":"ACTION_TYPE","value":"ACCOUNT_WIPE"},{"name":"DEVICE_ID","value":"id"},{"name":"DEVICE_TYPE","value":"type"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ACTION_CANCELLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['user', 'info'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.mobile.action.id': 'id', 'gsuite.admin.mobile.action.type': 'ACCOUNT_WIPE', 'gsuite.admin.device.id': 'id', 'gsuite.admin.device.type': 'type', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'MOBILE_SETTINGS'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_004_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'url.path': '/path/in/url', 'url.full': 'http://example.com/path/in/url', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"SITES_SETTINGS","name":"ADD_WEB_ADDRESS","parameters":[{"name":"SITE_LOCATION","value":"/path/in/url"},{"name":"WEB_ADDRESS","value":"http://example.com/path/in/url"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ADD_WEB_ADDRESS', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['creation'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.event.type': 'SITES_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_004_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_004_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-sites-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '95b159ac-a0ba-4abe-8a09-e2b7ccda6ab6', 'hostnam....9.0'}, 'event': {'action': 'VIEW_SITE_DETAILS', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'url.path': '/path/in/url', 'url.full': 'http://example.com/path/in/url', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"SITES_SETTINGS","name":"ADD_WEB_ADDRESS","parameters":[{"name":"SITE_LOCATION","value":"/path/in/url"},{"name":"WEB_ADDRESS","value":"http://example.com/path/in/url"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ADD_WEB_ADDRESS', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['creation'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.event.type': 'SITES_SETTINGS'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_005_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DELEGATED_ADMIN_SETTINGS","name":"ASSIGN_ROLE","parameters":[{"name":"ORG_UNIT_NAME","value":"org"},{"name":"ROLE_NAME","value":"_DIRECTORY_SYNC_ADMIN_ROLE"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ASSIGN_ROLE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.role.name': '_DIRECTORY_SYNC_ADMIN_ROLE', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'DELEGATED_ADMIN_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_005_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_005_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-delegatedadmin-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '29665a92-bbcd-4323-a6f2-c0a20d9a3ef4', 'hostnam...1.9.0'}, 'event': {'action': 'RENAME_ROLE', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DELEGATED_ADMIN_SETTINGS","name":"ASSIGN_ROLE","parameters":[{"name":"ORG_UNIT_NAME","value":"org"},{"name":"ROLE_NAME","value":"_DIRECTORY_SYNC_ADMIN_ROLE"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ASSIGN_ROLE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.role.name': '_DIRECTORY_SYNC_ADMIN_ROLE', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'DELEGATED_ADMIN_SETTINGS'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_006_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"ORG_SETTINGS","name":"CHROME_LICENSES_ENABLED","parameters":[{"name":"APPLICATION_NAME","value":"app"},{"name":"CHROME_LICENSES_ENABLED","value":"DISABLED"},{"name":"ORG_UNIT_NAME","value":"org"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHROME_LICENSES_ENABLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'app', 'gsuite.admin.chrome_licenses.enabled': 'DISABLED', 'gsuite.event.type': 'ORG_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_006_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_006_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-org-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'e00feeed-c6ae-47a3-b022-d8067f3ebc32', 'hostnam..., 'event': {'action': 'ASSIGN_CUSTOM_LOGO', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"ORG_SETTINGS","name":"CHROME_LICENSES_ENABLED","parameters":[{"name":"APPLICATION_NAME","value":"app"},{"name":"CHROME_LICENSES_ENABLED","value":"DISABLED"},{"name":"ORG_UNIT_NAME","value":"org"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHROME_LICENSES_ENABLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'app', 'gsuite.admin.chrome_licenses.enabled': 'DISABLED', 'gsuite.event.type': 'ORG_SETTINGS'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_007_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"CHROME_OS_SETTINGS","name":"CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING","parameters":[{"name":"APP_ID","value":"2345"},{"name":"CHROME_OS_SESSION_TYPE","value":"type"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING', 'event.id': '1', 'event.category': ['iam', 'configuration'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.id': '2345', 'gsuite.admin.chrome_os.session_type': 'type', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'CHROME_OS_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_007_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_007_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-chromeos-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'b491490f-cef4-4f03-bc38-e0c91deb38b3', 'hostnam...CHROME_OS_DEVICE_SETTING', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"CHROME_OS_SETTINGS","name":"CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING","parameters":[{"name":"APP_ID","value":"2345"},{"name":"CHROME_OS_SESSION_TYPE","value":"type"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING', 'event.id': '1', 'event.category': ['iam', 'configuration'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.id': '2345', 'gsuite.admin.chrome_os.session_type': 'type', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'CHROME_OS_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_008_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DOCS_SETTINGS","name":"TRANSFER_DOCUMENT_OWNERSHIP","parameters":[{"name":"DOMAIN_NAME","value":"example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'TRANSFER_DOCUMENT_OWNERSHIP', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.domain.name': 'example.com', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.admin.new_value': 'new', 'gsuite.event.type': 'DOCS_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_008_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_008_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-docs-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'b3bd56fe-f647-4033-a4cb-2ee16604873d', 'hostnam...action': 'CHANGE_DOCS_SETTING', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DOCS_SETTINGS","name":"TRANSFER_DOCUMENT_OWNERSHIP","parameters":[{"name":"DOMAIN_NAME","value":"example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'TRANSFER_DOCUMENT_OWNERSHIP', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.domain.name': 'example.com', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.admin.new_value': 'new', 'gsuite.event.type': 'DOCS_SETTINGS'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_009_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"EMAIL_SETTINGS","name":"DROP_FROM_QUARANTINE","parameters":[{"name":"EMAIL_LOG_SEARCH_MSG_ID","value":"id"},{"name":"QUARANTINE_NAME","value":"quarantine"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'DROP_FROM_QUARANTINE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.email.log_search_filter.message_id': 'id', 'gsuite.admin.email.quarantine_name': 'quarantine', 'gsuite.event.type': 'EMAIL_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_009_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_009_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-gmail-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'd48550ce-5f27-42bf-aca8-8a1ef32262fe', 'hostnam...': 'CREATE_GMAIL_SETTING', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"EMAIL_SETTINGS","name":"DROP_FROM_QUARANTINE","parameters":[{"name":"EMAIL_LOG_SEARCH_MSG_ID","value":"id"},{"name":"QUARANTINE_NAME","value":"quarantine"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'DROP_FROM_QUARANTINE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.email.log_search_filter.message_id': 'id', 'gsuite.admin.email.quarantine_name': 'quarantine', 'gsuite.event.type': 'EMAIL_SETTINGS'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

`Build&Test / x-pack/filebeat-build / test_fileset_file_010_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest`

Expand to view the error details

 AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"APPLICATION_SETTINGS","name":"CHANGE_APPLICATION_SETTING","parameters":[{"name":"APPLICATION_EDITION","value":"basic"},{"name":"APPLICATION_NAME","value":"drive"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_APPLICATION_SETTING', 'event.id': '1', 'event.type': ['change'], 'event.category': ['iam', 'configuration'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'drive', 'gsuite.admin.application.edition': 'basic', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'APPLICATION_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

Expand to view the stacktrace

 a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_010_gsuite>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
    cfgfile=cfgfile)
../../filebeat/tests/system/test_modules.py:193: in run_on_file
    self._test_expected_events(test_file, objects)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_010_gsuite>
test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-application-test.json.log'
objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '2a969495-4dbe-4993-ad4e-1e452dae175a', 'hostnam... {'action': 'CREATE_MANAGED_CONFIGURATION', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]

    def _test_expected_events(self, test_file, objects):
    
        # Generate expected files if GENERATE env variable is set
        if os.getenv("GENERATE"):
            with open(test_file + "-expected.json", 'w') as f:
                # Flatten an cleanup objects
                # This makes sure when generated on different machines / version the expected.json stays the same.
                for k, obj in enumerate(objects):
                    objects[k] = self.flatten_object(obj, {}, "")
                    clean_keys(objects[k])
    
                json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
    
        with open(test_file + "-expected.json", "r") as f:
            expected = json.load(f)
    
        assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
            len(expected), len(objects))
    
        for idx in range(len(expected)):
            ev = expected[idx]
            obj = objects[idx]
    
            # Flatten objects for easier comparing
            obj = self.flatten_object(obj, {}, "")
            clean_keys(obj)
            clean_keys(ev)
    
            d = DeepDiff(ev, obj, ignore_order=True)
    
>           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
E           AssertionError: The following expected object doesn't match:
E              Diff:
E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"APPLICATION_SETTINGS","name":"CHANGE_APPLICATION_SETTING","parameters":[{"name":"APPLICATION_EDITION","value":"basic"},{"name":"APPLICATION_NAME","value":"drive"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_APPLICATION_SETTING', 'event.id': '1', 'event.type': ['change'], 'event.category': ['iam', 'configuration'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'drive', 'gsuite.admin.application.edition': 'basic', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'APPLICATION_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'}
E           assert 1 == 0
E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})

../../filebeat/tests/system/test_modules.py:225: AssertionError

Steps errors

Expand to view the steps failures

`x-pack/filebeat-build - mage build test`

Took 37 min 6 sec . View more details on here
Description: mage build test

`Error signal`

Took 0 min 0 sec . View more details on here
Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-04-23T13:05:08.605Z]   Go version:       go1.13.15
[2021-04-23T13:05:08.605Z]   Git commit:       46229ca
[2021-04-23T13:05:08.605Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-04-23T13:05:08.605Z]   OS/Arch:          linux/amd64
[2021-04-23T13:05:08.605Z]   Experimental:     false
[2021-04-23T13:05:08.605Z]  containerd:
[2021-04-23T13:05:08.605Z]   Version:          1.4.4
[2021-04-23T13:05:08.605Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-04-23T13:05:08.605Z]  runc:
[2021-04-23T13:05:08.605Z]   Version:          1.0.0-rc93
[2021-04-23T13:05:08.605Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-04-23T13:05:08.605Z]  docker-init:
[2021-04-23T13:05:08.605Z]   Version:          0.19.0
[2021-04-23T13:05:08.605Z]   GitCommit:        de40ad0
[2021-04-23T13:05:08.605Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-04-23T13:05:08.605Z] Unable to find image 'alpine:3.4' locally
[2021-04-23T13:05:09.540Z] 3.4: Pulling from library/alpine
[2021-04-23T13:05:09.540Z] c1e54eec4b57: Pulling fs layer
[2021-04-23T13:05:09.799Z] c1e54eec4b57: Download complete
[2021-04-23T13:05:10.061Z] c1e54eec4b57: Pull complete
[2021-04-23T13:05:10.061Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-04-23T13:05:10.061Z] Status: Downloaded newer image for alpine:3.4
[2021-04-23T13:05:11.964Z] Change permissions with write access of all files inside the specific folder
[2021-04-23T13:05:13.383Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build
[2021-04-23T13:05:13.721Z] + rm -rf ve
[2021-04-23T13:05:13.721Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-04-23T13:05:14.136Z] + python .ci/scripts/pre_archive_test.py
[2021-04-23T13:05:16.670Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-04-23T13:05:16.706Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build
[2021-04-23T13:05:16.756Z] Recording test results
[2021-04-23T13:05:18.492Z] [Checks API] No suitable checks publisher found.
[2021-04-23T13:05:19.036Z] + tar --version
[2021-04-23T13:05:19.568Z] + tar --exclude=test-build-artifacts-x-pack-filebeat-build-tgz -czf test-build-artifacts-x-pack-filebeat-build-tgz .
[2021-04-23T13:06:16.454Z] + gsutil --version
[2021-04-23T13:06:16.455Z] /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build@tmp/durable-7acd6a97/script.sh: 1: /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build@tmp/durable-7acd6a97/script.sh: gsutil: not found
[2021-04-23T13:06:16.525Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build@tmp
[2021-04-23T13:06:16.892Z] + wget --version
[2021-04-23T13:06:17.274Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-04-23T13:06:17.913Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-04-23T13:06:20.616Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-04-23T13:06:21.048Z] + gcloud auth activate-service-account --key-file ****
[2021-04-23T13:06:22.654Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-04-23T13:06:23.141Z] + gsutil -m -q cp -a public-read test-build-artifacts-x-pack-filebeat-build-tgz gs://beats-ci-temp/Beats/beats/PR-25271-1
[2021-04-23T13:06:28.909Z] + python .ci/scripts/search_system_tests.py
[2021-04-23T13:06:28.952Z] [INFO] system-tests='build/x-pack/filebeat/build/system-tests'. If no empty then let's create a tarball
[2021-04-23T13:06:29.312Z] + tar --version
[2021-04-23T13:06:29.689Z] + tar --exclude=x-pack-filebeat--system-tests-linux-tgz -czf x-pack-filebeat--system-tests-linux-tgz build/x-pack/filebeat/build/system-tests
[2021-04-23T13:07:08.832Z] + gsutil --version
[2021-04-23T13:07:08.832Z] /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats@tmp/durable-f527c42b/script.sh: 1: /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats@tmp/durable-f527c42b/script.sh: gsutil: not found
[2021-04-23T13:07:08.904Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats@tmp
[2021-04-23T13:07:09.331Z] + wget --version
[2021-04-23T13:07:09.721Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-04-23T13:07:10.374Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-04-23T13:07:13.018Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-04-23T13:07:13.403Z] + gcloud auth activate-service-account --key-file ****
[2021-04-23T13:07:14.891Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-04-23T13:07:15.297Z] + gsutil -m -q cp -a public-read x-pack-filebeat--system-tests-linux-tgz gs://beats-ci-temp/Beats/beats/PR-25271-1
[2021-04-23T13:07:19.158Z] + go clean -modcache
[2021-04-23T13:07:22.814Z] Cleaning up /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb
[2021-04-23T13:07:22.814Z] Client: Docker Engine - Community
[2021-04-23T13:07:22.814Z]  Version:           20.10.3
[2021-04-23T13:07:22.814Z]  API version:       1.41
[2021-04-23T13:07:22.814Z]  Go version:        go1.13.15
[2021-04-23T13:07:22.814Z]  Git commit:        48d30b5
[2021-04-23T13:07:22.814Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-04-23T13:07:22.814Z]  OS/Arch:           linux/amd64
[2021-04-23T13:07:22.814Z]  Context:           default
[2021-04-23T13:07:22.814Z]  Experimental:      true
[2021-04-23T13:07:22.814Z] 
[2021-04-23T13:07:22.814Z] Server: Docker Engine - Community
[2021-04-23T13:07:22.814Z]  Engine:
[2021-04-23T13:07:22.814Z]   Version:          20.10.3
[2021-04-23T13:07:22.814Z]   API version:      1.41 (minimum version 1.12)
[2021-04-23T13:07:22.814Z]   Go version:       go1.13.15
[2021-04-23T13:07:22.814Z]   Git commit:       46229ca
[2021-04-23T13:07:22.814Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-04-23T13:07:22.814Z]   OS/Arch:          linux/amd64
[2021-04-23T13:07:22.814Z]   Experimental:     false
[2021-04-23T13:07:22.814Z]  containerd:
[2021-04-23T13:07:22.814Z]   Version:          1.4.4
[2021-04-23T13:07:22.814Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-04-23T13:07:22.814Z]  runc:
[2021-04-23T13:07:22.814Z]   Version:          1.0.0-rc93
[2021-04-23T13:07:22.814Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-04-23T13:07:22.814Z]  docker-init:
[2021-04-23T13:07:22.814Z]   Version:          0.19.0
[2021-04-23T13:07:22.814Z]   GitCommit:        de40ad0
[2021-04-23T13:07:22.814Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-04-23T13:07:27.102Z] Change permissions with write access of all files inside the specific folder
[2021-04-23T13:07:27.153Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb
[2021-04-23T13:07:33.777Z] Failed in branch x-pack/filebeat-build
[2021-04-23T13:07:34.478Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-04-23T13:07:34.621Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-25271/src/github.com/elastic/beats
[2021-04-23T13:07:35.232Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-25271
[2021-04-23T13:07:35.312Z] [INFO] getVaultSecret: Getting secrets
[2021-04-23T13:07:35.373Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-04-23T13:07:36.469Z] + chmod 755 generate-build-data.sh
[2021-04-23T13:07:36.469Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/runs/1 FAILURE 3643526
[2021-04-23T13:07:36.469Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/runs/1/steps/?limit=10000 -o steps-info.json
[2021-04-23T13:07:38.331Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/runs/1/tests/?status=FAILED -o tests-errors.json

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	150
Passed	13523
Skipped	2299
Total	15972

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / x-pack/filebeat-build / test_fileset_file_001_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_002_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_003_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_004_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_005_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_006_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_007_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_008_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_009_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_010_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_011_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_012_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_013_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_014_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_015_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_016_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_017_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_018_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_019_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_020_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_021_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_024_azure – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_027_azure – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_032_fortinet – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_036_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_037_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_038_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_039_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_040_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_046_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_053_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_059_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_060_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_061_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_064_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_065_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_068_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_069_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_077_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_079_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_082_checkpoint – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_083_sonicwall – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_093_gcp – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_095_gcp – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_113_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_114_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_115_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_116_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_117_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_119_squid – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_123_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_124_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_128_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_131_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_132_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_135_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_136_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_138_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_141_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_142_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_143_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_144_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_145_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_146_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_147_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_148_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_149_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_150_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_151_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_152_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_153_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_154_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_155_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_156_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_157_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_158_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_159_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_160_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_161_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_167_iptables – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_169_okta – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_172_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_173_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_176_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_177_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_178_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_180_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_182_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_186_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_190_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_192_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_193_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_195_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_198_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_203_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_205_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_208_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_210_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_212_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_219_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels