Skip to content

Update expected log files of Filebeat modules#25271

Closed
kvch wants to merge 1 commit intoelastic:7.13from
kvch:fix-filebeat-update-module-expected-logs
Closed

Update expected log files of Filebeat modules#25271
kvch wants to merge 1 commit intoelastic:7.13from
kvch:fix-filebeat-update-module-expected-logs

Conversation

@kvch
Copy link
Copy Markdown
Contributor

@kvch kvch commented Apr 23, 2021

No description provided.

@botelastic botelastic Bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 23, 2021
@botelastic
Copy link
Copy Markdown

botelastic Bot commented Apr 23, 2021

This pull request doesn't have a Team:<team> label.

@kvch
Copy link
Copy Markdown
Contributor Author

kvch commented Apr 23, 2021

Closing in favour of #25272

@kvch kvch closed this Apr 23, 2021
@elasticmachine
Copy link
Copy Markdown
Contributor

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #25271 opened

  • Start Time: 2021-04-23T12:06:52.659+0000

  • Duration: 61 min 44 sec

  • Commit: 5a49a0e

Test stats 🧪

Test Results
Failed 150
Passed 13523
Skipped 2299
Total 15972

Trends 🧪

Image of Build Times

Image of Tests

Test errors 150

Expand to view the tests failures

> Show only the first 10 test failures

Build&Test / x-pack/filebeat-build / test_fileset_file_001_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'saml', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'google_workspace.saml.initiated_by': 'idp', 'google_workspace.saml.status_code': 'SUCCESS_URI', 'google_workspace.saml.application_name': 'app', 'google_workspace.saml.second_level_status_code': 'SUCCESS_URI', 'google_workspace.saml.failure_type': 'failure_app_not_configured_for_user', 'google_workspace.saml.orgunit_path': 'ounit', 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"saml","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"login","name":"login_failure","parameters":[{"name":"application_name","value":"app"},{"name":"failure_type","value":"failure_app_not_configured_for_user"},{"name":"initiated_by","value":"idp"},{"name":"orgunit_path","value":"ounit"},{"name":"saml_second_level_status_code","value":"SUCCESS_URI"},{"name":"saml_status_code","value":"SUCCESS_URI"}]}}', 'event.provider': 'saml', 'event.module': 'gsuite', 'event.action': 'login_failure', 'event.id': '1', 'event.type': ['start'], 'event.category': ['authentication', 'session'], 'event.dataset': 'gsuite.saml', 'event.outcome': 'failure', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.event.type': 'login'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_001_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_001_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/saml/test/gsuite-saml-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '41e8efbc-fac2-4392-bbe3-2e41581761c2', 'hostnam...{'action': 'login_success', 'category': ['authentication', 'session'], 'dataset': 'gsuite.saml', 'id': '1', ...}, ...}]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'saml', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'google_workspace.saml.initiated_by': 'idp', 'google_workspace.saml.status_code': 'SUCCESS_URI', 'google_workspace.saml.application_name': 'app', 'google_workspace.saml.second_level_status_code': 'SUCCESS_URI', 'google_workspace.saml.failure_type': 'failure_app_not_configured_for_user', 'google_workspace.saml.orgunit_path': 'ounit', 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"saml","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"login","name":"login_failure","parameters":[{"name":"application_name","value":"app"},{"name":"failure_type","value":"failure_app_not_configured_for_user"},{"name":"initiated_by","value":"idp"},{"name":"orgunit_path","value":"ounit"},{"name":"saml_second_level_status_code","value":"SUCCESS_URI"},{"name":"saml_status_code","value":"SUCCESS_URI"}]}}', 'event.provider': 'saml', 'event.module': 'gsuite', 'event.action': 'login_failure', 'event.id': '1', 'event.type': ['start'], 'event.category': ['authentication', 'session'], 'event.dataset': 'gsuite.saml', 'event.outcome': 'failure', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.event.type': 'login'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_002_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'groups', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"groups","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"acl_change","name":"change_acl_permission","parameters":[{"name":"acl_permission","value":"can_add_members"},{"name":"group_email","value":"group@example.com"},{"name":"new_value_repeated","multiValue":["managers","members"]},{"name":"old_value_repeated","multiValue":["managers"]}]}}', 'event.provider': 'groups', 'event.module': 'gsuite', 'event.action': 'change_acl_permission', 'event.id': '1', 'event.type': ['group', 'change'], 'event.category': ['iam'], 'event.dataset': 'gsuite.groups', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'group.domain': 'example.com', 'group.name': 'group', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.groups.old_value': ['managers'], 'gsuite.groups.email': 'group@example.com', 'gsuite.groups.acl_permission': 'can_add_members', 'gsuite.groups.new_value': ['managers', 'members'], 'gsuite.event.type': 'acl_change'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_002_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_002_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/groups/test/gsuite-groups-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '35a9a354-2216-48ca-9413-766f848f44bb', 'hostnam...: 'change_basic_setting', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.groups', 'id': '1', ...}, ...}, ...]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'groups', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"groups","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"acl_change","name":"change_acl_permission","parameters":[{"name":"acl_permission","value":"can_add_members"},{"name":"group_email","value":"group@example.com"},{"name":"new_value_repeated","multiValue":["managers","members"]},{"name":"old_value_repeated","multiValue":["managers"]}]}}', 'event.provider': 'groups', 'event.module': 'gsuite', 'event.action': 'change_acl_permission', 'event.id': '1', 'event.type': ['group', 'change'], 'event.category': ['iam'], 'event.dataset': 'gsuite.groups', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'group.domain': 'example.com', 'group.name': 'group', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.groups.old_value': ['managers'], 'gsuite.groups.email': 'group@example.com', 'gsuite.groups.acl_permission': 'can_add_members', 'gsuite.groups.new_value': ['managers', 'members'], 'gsuite.event.type': 'acl_change'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_003_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"MOBILE_SETTINGS","name":"ACTION_CANCELLED","parameters":[{"name":"ACTION_ID","value":"id"},{"name":"ACTION_TYPE","value":"ACCOUNT_WIPE"},{"name":"DEVICE_ID","value":"id"},{"name":"DEVICE_TYPE","value":"type"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ACTION_CANCELLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['user', 'info'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.mobile.action.id': 'id', 'gsuite.admin.mobile.action.type': 'ACCOUNT_WIPE', 'gsuite.admin.device.id': 'id', 'gsuite.admin.device.type': 'type', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'MOBILE_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_003_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_003_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-mobile-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '87de6457-fdf5-4d59-af8e-7526a8d0a62a', 'hostnam...ent': {'action': 'COMPANY_DEVICE_DELETION', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"MOBILE_SETTINGS","name":"ACTION_CANCELLED","parameters":[{"name":"ACTION_ID","value":"id"},{"name":"ACTION_TYPE","value":"ACCOUNT_WIPE"},{"name":"DEVICE_ID","value":"id"},{"name":"DEVICE_TYPE","value":"type"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ACTION_CANCELLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['user', 'info'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.mobile.action.id': 'id', 'gsuite.admin.mobile.action.type': 'ACCOUNT_WIPE', 'gsuite.admin.device.id': 'id', 'gsuite.admin.device.type': 'type', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'MOBILE_SETTINGS'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_004_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'url.path': '/path/in/url', 'url.full': 'http://example.com/path/in/url', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"SITES_SETTINGS","name":"ADD_WEB_ADDRESS","parameters":[{"name":"SITE_LOCATION","value":"/path/in/url"},{"name":"WEB_ADDRESS","value":"http://example.com/path/in/url"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ADD_WEB_ADDRESS', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['creation'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.event.type': 'SITES_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_004_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_004_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-sites-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '95b159ac-a0ba-4abe-8a09-e2b7ccda6ab6', 'hostnam....9.0'}, 'event': {'action': 'VIEW_SITE_DETAILS', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'url.path': '/path/in/url', 'url.full': 'http://example.com/path/in/url', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"SITES_SETTINGS","name":"ADD_WEB_ADDRESS","parameters":[{"name":"SITE_LOCATION","value":"/path/in/url"},{"name":"WEB_ADDRESS","value":"http://example.com/path/in/url"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ADD_WEB_ADDRESS', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['creation'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.event.type': 'SITES_SETTINGS'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_005_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DELEGATED_ADMIN_SETTINGS","name":"ASSIGN_ROLE","parameters":[{"name":"ORG_UNIT_NAME","value":"org"},{"name":"ROLE_NAME","value":"_DIRECTORY_SYNC_ADMIN_ROLE"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ASSIGN_ROLE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.role.name': '_DIRECTORY_SYNC_ADMIN_ROLE', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'DELEGATED_ADMIN_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_005_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_005_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-delegatedadmin-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '29665a92-bbcd-4323-a6f2-c0a20d9a3ef4', 'hostnam...1.9.0'}, 'event': {'action': 'RENAME_ROLE', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DELEGATED_ADMIN_SETTINGS","name":"ASSIGN_ROLE","parameters":[{"name":"ORG_UNIT_NAME","value":"org"},{"name":"ROLE_NAME","value":"_DIRECTORY_SYNC_ADMIN_ROLE"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'ASSIGN_ROLE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.role.name': '_DIRECTORY_SYNC_ADMIN_ROLE', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.event.type': 'DELEGATED_ADMIN_SETTINGS'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_006_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"ORG_SETTINGS","name":"CHROME_LICENSES_ENABLED","parameters":[{"name":"APPLICATION_NAME","value":"app"},{"name":"CHROME_LICENSES_ENABLED","value":"DISABLED"},{"name":"ORG_UNIT_NAME","value":"org"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHROME_LICENSES_ENABLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'app', 'gsuite.admin.chrome_licenses.enabled': 'DISABLED', 'gsuite.event.type': 'ORG_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_006_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_006_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-org-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'e00feeed-c6ae-47a3-b022-d8067f3ebc32', 'hostnam..., 'event': {'action': 'ASSIGN_CUSTOM_LOGO', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"ORG_SETTINGS","name":"CHROME_LICENSES_ENABLED","parameters":[{"name":"APPLICATION_NAME","value":"app"},{"name":"CHROME_LICENSES_ENABLED","value":"DISABLED"},{"name":"ORG_UNIT_NAME","value":"org"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHROME_LICENSES_ENABLED', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'app', 'gsuite.admin.chrome_licenses.enabled': 'DISABLED', 'gsuite.event.type': 'ORG_SETTINGS'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_007_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"CHROME_OS_SETTINGS","name":"CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING","parameters":[{"name":"APP_ID","value":"2345"},{"name":"CHROME_OS_SESSION_TYPE","value":"type"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING', 'event.id': '1', 'event.category': ['iam', 'configuration'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.id': '2345', 'gsuite.admin.chrome_os.session_type': 'type', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'CHROME_OS_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_007_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_007_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-chromeos-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'b491490f-cef4-4f03-bc38-e0c91deb38b3', 'hostnam...CHROME_OS_DEVICE_SETTING', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"CHROME_OS_SETTINGS","name":"CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING","parameters":[{"name":"APP_ID","value":"2345"},{"name":"CHROME_OS_SESSION_TYPE","value":"type"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_CHROME_OS_ANDROID_APPLICATION_SETTING', 'event.id': '1', 'event.category': ['iam', 'configuration'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.id': '2345', 'gsuite.admin.chrome_os.session_type': 'type', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'CHROME_OS_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_008_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DOCS_SETTINGS","name":"TRANSFER_DOCUMENT_OWNERSHIP","parameters":[{"name":"DOMAIN_NAME","value":"example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'TRANSFER_DOCUMENT_OWNERSHIP', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.domain.name': 'example.com', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.admin.new_value': 'new', 'gsuite.event.type': 'DOCS_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_008_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_008_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-docs-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'b3bd56fe-f647-4033-a4cb-2ee16604873d', 'hostnam...action': 'CHANGE_DOCS_SETTING', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo', 'user'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"DOCS_SETTINGS","name":"TRANSFER_DOCUMENT_OWNERSHIP","parameters":[{"name":"DOMAIN_NAME","value":"example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"USER_EMAIL","value":"user@example.com"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'TRANSFER_DOCUMENT_OWNERSHIP', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'user.target.domain': 'example.com', 'user.target.name': 'user', 'user.target.email': 'user@example.com', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.domain.name': 'example.com', 'gsuite.admin.user.email': 'user@example.com', 'gsuite.admin.new_value': 'new', 'gsuite.event.type': 'DOCS_SETTINGS'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_009_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"EMAIL_SETTINGS","name":"DROP_FROM_QUARANTINE","parameters":[{"name":"EMAIL_LOG_SEARCH_MSG_ID","value":"id"},{"name":"QUARANTINE_NAME","value":"quarantine"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'DROP_FROM_QUARANTINE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.email.log_search_filter.message_id': 'id', 'gsuite.admin.email.quarantine_name': 'quarantine', 'gsuite.event.type': 'EMAIL_SETTINGS'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_009_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_009_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-gmail-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': 'd48550ce-5f27-42bf-aca8-8a1ef32262fe', 'hostnam...': 'CREATE_GMAIL_SETTING', 'category': ['iam', 'configuration'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"EMAIL_SETTINGS","name":"DROP_FROM_QUARANTINE","parameters":[{"name":"EMAIL_LOG_SEARCH_MSG_ID","value":"id"},{"name":"QUARANTINE_NAME","value":"quarantine"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'DROP_FROM_QUARANTINE', 'event.id': '1', 'event.category': ['iam'], 'event.type': ['change'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.email.log_search_filter.message_id': 'id', 'gsuite.admin.email.quarantine_name': 'quarantine', 'gsuite.event.type': 'EMAIL_SETTINGS'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Build&Test / x-pack/filebeat-build / test_fileset_file_010_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
    Expand to view the error details

     AssertionError: The following expected object doesn't match:    Diff:   {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object:    {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"APPLICATION_SETTINGS","name":"CHANGE_APPLICATION_SETTING","parameters":[{"name":"APPLICATION_EDITION","value":"basic"},{"name":"APPLICATION_NAME","value":"drive"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_APPLICATION_SETTING', 'event.id': '1', 'event.type': ['change'], 'event.category': ['iam', 'configuration'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'drive', 'gsuite.admin.application.edition': 'basic', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'APPLICATION_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'} assert 1 == 0  +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}) 
    

    Expand to view the stacktrace

     a = (<test_xpack_modules.XPackTest testMethod=test_fileset_file_010_gsuite>,)
    
        @wraps(func)
        def standalone_func(*a):
    >       return func(*(a + p.args), **p.kwargs)
    
    ../../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    ../../filebeat/tests/system/test_modules.py:107: in test_fileset_file
        cfgfile=cfgfile)
    ../../filebeat/tests/system/test_modules.py:193: in run_on_file
        self._test_expected_events(test_file, objects)
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <test_xpack_modules.XPackTest testMethod=test_fileset_file_010_gsuite>
    test_file = '/go/src/github.com/elastic/beats/x-pack/filebeat/module/gsuite/admin/test/gsuite-admin-application-test.json.log'
    objects = [{'@timestamp': '2020-10-02T15:00:00.000Z', 'agent': {'ephemeral_id': '2a969495-4dbe-4993-ad4e-1e452dae175a', 'hostnam... {'action': 'CREATE_MANAGED_CONFIGURATION', 'category': ['iam'], 'dataset': 'gsuite.admin', 'id': '1', ...}, ...}, ...]
    
        def _test_expected_events(self, test_file, objects):
        
            # Generate expected files if GENERATE env variable is set
            if os.getenv("GENERATE"):
                with open(test_file + "-expected.json", 'w') as f:
                    # Flatten an cleanup objects
                    # This makes sure when generated on different machines / version the expected.json stays the same.
                    for k, obj in enumerate(objects):
                        objects[k] = self.flatten_object(obj, {}, "")
                        clean_keys(objects[k])
        
                    json.dump(objects, f, indent=4, separators=(',', ': '), sort_keys=True)
        
            with open(test_file + "-expected.json", "r") as f:
                expected = json.load(f)
        
            assert len(expected) == len(objects), "expected {} events to compare but got {}".format(
                len(expected), len(objects))
        
            for idx in range(len(expected)):
                ev = expected[idx]
                obj = objects[idx]
        
                # Flatten objects for easier comparing
                obj = self.flatten_object(obj, {}, "")
                clean_keys(obj)
                clean_keys(ev)
        
                d = DeepDiff(ev, obj, ignore_order=True)
        
    >           assert len(d) == 0, "The following expected object doesn't match:\n Diff:\n{}, full object: \n{}".format(d, obj)
    E           AssertionError: The following expected object doesn't match:
    E              Diff:
    E             {'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}}, full object: 
    E             {'log.offset': 0, 'source.geo.continent_name': 'North America', 'source.geo.region_iso_code': 'US-PA', 'source.geo.city_name': 'State College', 'source.geo.country_iso_code': 'US', 'source.geo.country_name': 'United States', 'source.geo.region_name': 'Pennsylvania', 'source.geo.location.lon': -77.8618, 'source.geo.location.lat': 40.7957, 'source.as.number': 7922, 'source.as.organization.name': 'COMCAST-7922', 'source.ip': '98.235.162.24', 'source.user.domain': 'bar.com', 'source.user.name': 'foo', 'source.user.id': '1', 'source.user.email': 'foo@bar.com', 'fileset.name': 'admin', 'tags': ['forwarded'], 'input.type': 'log', 'related.ip': ['98.235.162.24'], 'related.user': ['foo'], 'service.type': 'gsuite', 'organization.id': '1', 'event.original': '{"kind":"admin#reports#activity","id":{"time":"2020-10-02T15:00:00Z","uniqueQualifier":1,"applicationName":"admin","customerId":"1"},"actor":{"callerType":"USER","email":"foo@bar.com","profileId":1},"ownerDomain":"elastic.com","ipAddress":"98.235.162.24","events":{"type":"APPLICATION_SETTINGS","name":"CHANGE_APPLICATION_SETTING","parameters":[{"name":"APPLICATION_EDITION","value":"basic"},{"name":"APPLICATION_NAME","value":"drive"},{"name":"GROUP_EMAIL","value":"group@example.com"},{"name":"NEW_VALUE","value":"new"},{"name":"OLD_VALUE","value":"old"},{"name":"ORG_UNIT_NAME","value":"org"},{"name":"SETTING_NAME","value":"setting"}]}}', 'event.provider': 'admin', 'event.module': 'gsuite', 'event.action': 'CHANGE_APPLICATION_SETTING', 'event.id': '1', 'event.type': ['change'], 'event.category': ['iam', 'configuration'], 'event.dataset': 'gsuite.admin', 'user.domain': 'bar.com', 'user.name': 'foo', 'user.id': '1', 'gsuite.actor.type': 'USER', 'gsuite.kind': 'admin#reports#activity', 'gsuite.organization.domain': 'elastic.com', 'gsuite.admin.org_unit.name': 'org', 'gsuite.admin.application.name': 'drive', 'gsuite.admin.application.edition': 'basic', 'gsuite.admin.old_value': 'old', 'gsuite.admin.new_value': 'new', 'gsuite.admin.setting.name': 'setting', 'gsuite.admin.group.email': 'group@example.com', 'gsuite.event.type': 'APPLICATION_SETTINGS', 'group.domain': 'example.com', 'group.name': 'group'}
    E           assert 1 == 0
    E            +  where 1 = len({'values_changed': {"root['source.as.organization.name']": {'new_value': 'COMCAST-7922', 'old_value': 'Comcast Cable Communications, LLC'}}})
    
    ../../filebeat/tests/system/test_modules.py:225: AssertionError 
    

Steps errors 2

Expand to view the steps failures

x-pack/filebeat-build - mage build test
  • Took 37 min 6 sec . View more details on here
  • Description: mage build test
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-04-23T13:05:08.605Z]   Go version:       go1.13.15
[2021-04-23T13:05:08.605Z]   Git commit:       46229ca
[2021-04-23T13:05:08.605Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-04-23T13:05:08.605Z]   OS/Arch:          linux/amd64
[2021-04-23T13:05:08.605Z]   Experimental:     false
[2021-04-23T13:05:08.605Z]  containerd:
[2021-04-23T13:05:08.605Z]   Version:          1.4.4
[2021-04-23T13:05:08.605Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-04-23T13:05:08.605Z]  runc:
[2021-04-23T13:05:08.605Z]   Version:          1.0.0-rc93
[2021-04-23T13:05:08.605Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-04-23T13:05:08.605Z]  docker-init:
[2021-04-23T13:05:08.605Z]   Version:          0.19.0
[2021-04-23T13:05:08.605Z]   GitCommit:        de40ad0
[2021-04-23T13:05:08.605Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-04-23T13:05:08.605Z] Unable to find image 'alpine:3.4' locally
[2021-04-23T13:05:09.540Z] 3.4: Pulling from library/alpine
[2021-04-23T13:05:09.540Z] c1e54eec4b57: Pulling fs layer
[2021-04-23T13:05:09.799Z] c1e54eec4b57: Download complete
[2021-04-23T13:05:10.061Z] c1e54eec4b57: Pull complete
[2021-04-23T13:05:10.061Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-04-23T13:05:10.061Z] Status: Downloaded newer image for alpine:3.4
[2021-04-23T13:05:11.964Z] Change permissions with write access of all files inside the specific folder
[2021-04-23T13:05:13.383Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build
[2021-04-23T13:05:13.721Z] + rm -rf ve
[2021-04-23T13:05:13.721Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-04-23T13:05:14.136Z] + python .ci/scripts/pre_archive_test.py
[2021-04-23T13:05:16.670Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-04-23T13:05:16.706Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build
[2021-04-23T13:05:16.756Z] Recording test results
[2021-04-23T13:05:18.492Z] [Checks API] No suitable checks publisher found.
[2021-04-23T13:05:19.036Z] + tar --version
[2021-04-23T13:05:19.568Z] + tar --exclude=test-build-artifacts-x-pack-filebeat-build-tgz -czf test-build-artifacts-x-pack-filebeat-build-tgz .
[2021-04-23T13:06:16.454Z] + gsutil --version
[2021-04-23T13:06:16.455Z] /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build@tmp/durable-7acd6a97/script.sh: 1: /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build@tmp/durable-7acd6a97/script.sh: gsutil: not found
[2021-04-23T13:06:16.525Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats/build@tmp
[2021-04-23T13:06:16.892Z] + wget --version
[2021-04-23T13:06:17.274Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-04-23T13:06:17.913Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-04-23T13:06:20.616Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-04-23T13:06:21.048Z] + gcloud auth activate-service-account --key-file ****
[2021-04-23T13:06:22.654Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-04-23T13:06:23.141Z] + gsutil -m -q cp -a public-read test-build-artifacts-x-pack-filebeat-build-tgz gs://beats-ci-temp/Beats/beats/PR-25271-1
[2021-04-23T13:06:28.909Z] + python .ci/scripts/search_system_tests.py
[2021-04-23T13:06:28.952Z] [INFO] system-tests='build/x-pack/filebeat/build/system-tests'. If no empty then let's create a tarball
[2021-04-23T13:06:29.312Z] + tar --version
[2021-04-23T13:06:29.689Z] + tar --exclude=x-pack-filebeat--system-tests-linux-tgz -czf x-pack-filebeat--system-tests-linux-tgz build/x-pack/filebeat/build/system-tests
[2021-04-23T13:07:08.832Z] + gsutil --version
[2021-04-23T13:07:08.832Z] /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats@tmp/durable-f527c42b/script.sh: 1: /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats@tmp/durable-f527c42b/script.sh: gsutil: not found
[2021-04-23T13:07:08.904Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb/src/github.com/elastic/beats@tmp
[2021-04-23T13:07:09.331Z] + wget --version
[2021-04-23T13:07:09.721Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-04-23T13:07:10.374Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-04-23T13:07:13.018Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-04-23T13:07:13.403Z] + gcloud auth activate-service-account --key-file ****
[2021-04-23T13:07:14.891Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-04-23T13:07:15.297Z] + gsutil -m -q cp -a public-read x-pack-filebeat--system-tests-linux-tgz gs://beats-ci-temp/Beats/beats/PR-25271-1
[2021-04-23T13:07:19.158Z] + go clean -modcache
[2021-04-23T13:07:22.814Z] Cleaning up /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb
[2021-04-23T13:07:22.814Z] Client: Docker Engine - Community
[2021-04-23T13:07:22.814Z]  Version:           20.10.3
[2021-04-23T13:07:22.814Z]  API version:       1.41
[2021-04-23T13:07:22.814Z]  Go version:        go1.13.15
[2021-04-23T13:07:22.814Z]  Git commit:        48d30b5
[2021-04-23T13:07:22.814Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-04-23T13:07:22.814Z]  OS/Arch:           linux/amd64
[2021-04-23T13:07:22.814Z]  Context:           default
[2021-04-23T13:07:22.814Z]  Experimental:      true
[2021-04-23T13:07:22.814Z] 
[2021-04-23T13:07:22.814Z] Server: Docker Engine - Community
[2021-04-23T13:07:22.814Z]  Engine:
[2021-04-23T13:07:22.814Z]   Version:          20.10.3
[2021-04-23T13:07:22.814Z]   API version:      1.41 (minimum version 1.12)
[2021-04-23T13:07:22.814Z]   Go version:       go1.13.15
[2021-04-23T13:07:22.814Z]   Git commit:       46229ca
[2021-04-23T13:07:22.814Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-04-23T13:07:22.814Z]   OS/Arch:          linux/amd64
[2021-04-23T13:07:22.814Z]   Experimental:     false
[2021-04-23T13:07:22.814Z]  containerd:
[2021-04-23T13:07:22.814Z]   Version:          1.4.4
[2021-04-23T13:07:22.814Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-04-23T13:07:22.814Z]  runc:
[2021-04-23T13:07:22.814Z]   Version:          1.0.0-rc93
[2021-04-23T13:07:22.814Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-04-23T13:07:22.814Z]  docker-init:
[2021-04-23T13:07:22.814Z]   Version:          0.19.0
[2021-04-23T13:07:22.814Z]   GitCommit:        de40ad0
[2021-04-23T13:07:22.814Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-04-23T13:07:27.102Z] Change permissions with write access of all files inside the specific folder
[2021-04-23T13:07:27.153Z] Running in /var/lib/jenkins/workspace/PR-25271-1-70622b48-a507-447c-923c-81ecde7e9afb
[2021-04-23T13:07:33.777Z] Failed in branch x-pack/filebeat-build
[2021-04-23T13:07:34.478Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-04-23T13:07:34.621Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-25271/src/github.com/elastic/beats
[2021-04-23T13:07:35.232Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-25271
[2021-04-23T13:07:35.312Z] [INFO] getVaultSecret: Getting secrets
[2021-04-23T13:07:35.373Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-04-23T13:07:36.469Z] + chmod 755 generate-build-data.sh
[2021-04-23T13:07:36.469Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/runs/1 FAILURE 3643526
[2021-04-23T13:07:36.469Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/runs/1/steps/?limit=10000 -o steps-info.json
[2021-04-23T13:07:38.331Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-25271/runs/1/tests/?status=FAILED -o tests-errors.json

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test Results
Failed 150
Passed 13523
Skipped 2299
Total 15972

Genuine test errors 100

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_001_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_002_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_003_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_004_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_005_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_006_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_007_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_008_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_009_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_010_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_011_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_012_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_013_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_014_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_015_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_016_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_017_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_018_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_019_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_020_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_021_gsuite – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_024_azure – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_027_azure – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_032_fortinet – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_036_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_037_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_038_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_039_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_040_juniper – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_046_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_053_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_059_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_060_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_061_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_064_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_065_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_068_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_069_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_077_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_079_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_082_checkpoint – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_083_sonicwall – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_093_gcp – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_095_gcp – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_113_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_114_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_115_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_116_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_117_panw – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_119_squid – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_123_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_124_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_128_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_131_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_132_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_135_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_136_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_138_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_141_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_142_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_143_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_144_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_145_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_146_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_147_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_148_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_149_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_150_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_151_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_152_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_153_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_154_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_155_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_156_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_157_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_158_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_159_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_160_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_161_google_workspace – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_167_iptables – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_169_okta – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_172_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_173_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_176_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_177_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_178_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_180_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_182_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_186_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_190_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_192_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_193_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_195_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_198_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_203_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_205_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_208_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_210_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_212_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
  • Name: Build&Test / x-pack/filebeat-build / test_fileset_file_219_cyberarkpas – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs_team Indicates that the issue/PR needs a Team:* label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants