From 178e073d697817776557340788c5125b6b0a50e5 Mon Sep 17 00:00:00 2001 From: yug-elastic Date: Fri, 30 Sep 2022 14:54:49 +0530 Subject: [PATCH 1/7] Implement 'setupaudittrail' data stream for Salesforce. --- .github/CODEOWNERS | 1 + packages/salesforce/_dev/build/build.yml | 3 + packages/salesforce/_dev/build/docs/README.md | 39 +++++ .../_dev/deploy/docker/docker-compose.yml | 15 ++ .../_dev/deploy/docker/files/config.yml | 53 +++++++ packages/salesforce/changelog.yml | 7 + .../_dev/test/pipeline/test-common-config.yml | 2 + .../test/pipeline/test-setupaudittrail.log | 1 + .../test-setupaudittrail.log-expected.json | 38 +++++ .../_dev/test/system/test-default-config.yml | 12 ++ .../agent/stream/httpjson.yml.hbs | 34 ++++ .../elasticsearch/ingest_pipeline/default.yml | 149 ++++++++++++++++++ .../setupaudittrail/fields/base-fields.yml | 15 ++ .../setupaudittrail/fields/ecs.yml | 28 ++++ .../setupaudittrail/fields/fields.yml | 30 ++++ .../data_stream/setupaudittrail/manifest.yml | 33 ++++ .../setupaudittrail/sample_event.json | 62 ++++++++ packages/salesforce/docs/README.md | 136 ++++++++++++++++ packages/salesforce/img/salesforce.svg | 25 +++ packages/salesforce/manifest.yml | 66 ++++++++ 20 files changed, 749 insertions(+) create mode 100644 packages/salesforce/_dev/build/build.yml create mode 100644 packages/salesforce/_dev/build/docs/README.md create mode 100644 packages/salesforce/_dev/deploy/docker/docker-compose.yml create mode 100644 packages/salesforce/_dev/deploy/docker/files/config.yml create mode 100644 packages/salesforce/changelog.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log create mode 100644 packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json create mode 100644 packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs create mode 100644 packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/fields/base-fields.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/fields/ecs.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/fields/fields.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/manifest.yml create mode 100644 packages/salesforce/data_stream/setupaudittrail/sample_event.json create mode 100644 packages/salesforce/docs/README.md create mode 100644 packages/salesforce/img/salesforce.svg create mode 100644 packages/salesforce/manifest.yml diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 48b5c703212..efb49eb50b1 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -148,6 +148,7 @@ /packages/radware @elastic/security-external-integrations /packages/redis @elastic/obs-service-integrations /packages/redisenterprise @elastic/obs-service-integrations +/packages/salesforce @elastic/obs-service-integrations /packages/santa @elastic/security-external-integrations /packages/security_detection_engine @elastic/protections /packages/sentinel_one @elastic/security-external-integrations diff --git a/packages/salesforce/_dev/build/build.yml b/packages/salesforce/_dev/build/build.yml new file mode 100644 index 00000000000..8d9e4bf7ac8 --- /dev/null +++ b/packages/salesforce/_dev/build/build.yml @@ -0,0 +1,3 @@ +dependencies: + ecs: + reference: git@v8.4.0 diff --git a/packages/salesforce/_dev/build/docs/README.md b/packages/salesforce/_dev/build/docs/README.md new file mode 100644 index 00000000000..b3ba0ee0bb5 --- /dev/null +++ b/packages/salesforce/_dev/build/docs/README.md @@ -0,0 +1,39 @@ +# Salesforce Integration + +## Overview + +The Salesforce integration allows you to monitor [Salesforce](https://www.salesforce.com/) instance. Salesforce provides customer relationship management service and also provides enterprise applications focused on customer service, marketing automation, analytics, and application development. + +Use the Salesforce integration to get visibility into the Salesforce Org operations and hold Salesforce accountable to the Service Level Agreements. Then visualize that data in Kibana, create alerts to notify you if something goes wrong, and reference logs when troubleshooting an issue. + +For example, if you want to check the number of successful and failed login attempts over time, you could check the same based on the ingested events or the visualization. Then you can create visualizations, alerts and troubleshoot by looking at the documents ingested in Elasticsearch. + +## Data streams + +The Salesforce integration collects log events using REST and Streaming API of Salesforce. + +**Logs** help you keep a record of events happening in Salesforce. +Log data streams collected by the Salesforce integration include [Login](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_login.htm) (using REST and Streaming API), [Logout](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_logout.htm) (using REST and Streaming API), [Apex](https://developer.salesforce.com/docs/atlas.en-us.238.0.object_reference.meta/object_reference/sforce_api_objects_apexclass.htm), and [SetupAuditTrail](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_setupaudittrail.htm). + +This integration uses: +- `httpjson` filebeat input to collect `login_rest`, `logout_rest`, `apex` and `setupaudittrail` events. +- `cometd` filebeat input to collect `login_stream` and `logout_stream` events. + +## Compatibility + +This integration has been tested against Salesforce API version `v54.0`. + +## Requirements + +You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. +You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware. + +## Logs reference + +### SetupAuditTrail + +This is the `setupaudittrail` data stream. It represents changes you or other admins made in your organization's Setup area for at least the last 180 days. + +{{event "setupaudittrail"}} + +{{fields "setupaudittrail"}} \ No newline at end of file diff --git a/packages/salesforce/_dev/deploy/docker/docker-compose.yml b/packages/salesforce/_dev/deploy/docker/docker-compose.yml new file mode 100644 index 00000000000..8694c1084de --- /dev/null +++ b/packages/salesforce/_dev/deploy/docker/docker-compose.yml @@ -0,0 +1,15 @@ +version: '2.3' +services: + salesforce: + image: docker.elastic.co/observability/stream:v0.8.0 + hostname: salesforce + ports: + - 8010 + volumes: + - ./files:/files:ro + environment: + PORT: 8010 + command: + - http-server + - --addr=:8010 + - --config=/files/config.yml diff --git a/packages/salesforce/_dev/deploy/docker/files/config.yml b/packages/salesforce/_dev/deploy/docker/files/config.yml new file mode 100644 index 00000000000..c623cfbafa1 --- /dev/null +++ b/packages/salesforce/_dev/deploy/docker/files/config.yml @@ -0,0 +1,53 @@ +rules: + - path: /services/oauth2/token + methods: ["POST"] + responses: + - status_code: 200 + headers: + content-type: ['application/json'] + body: '{"access_token":"access_token","instance_url":"https://temporary-intance-url","id":"https://login.salesforce.com/id/temp_id/temp_token","token_type":"Bearer","issued_at":"1633689089545","signature":"signature"}' + - path: /services/data/v54.0/query + methods: ["GET"] + query_params: + q: ["SELECT Id,CreatedDate,LogDate,LogFile FROM EventLogFile WHERE Interval = 'Hourly' AND EventType = 'Login' ORDER BY LogDate ASC NULLS FIRST"] + responses: + - status_code: 200 + body: '{"totalSize":1,"done":true,"records":[{"attributes":{"type":"EventLogFile","url":"/services/data/v54.0/sobjects/EventLogFile/0AT5j00002GVrfnGAD"},"Id":"0AT5j00002GVrfnGAD","CreatedDate":"2022-09-14T21:43:41.000+0000","LogDate":"2022-09-13T00:00:00.000+0000","LogFile":"/services/data/v54.0/sobjects/EventLogFile/0AT5j00002GVrfnGAD/LogFile"}]}' + headers: + content-type: ["text/json"] + - path: /services/data/v54.0/query + methods: ["GET"] + query_params: + q: ["SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,ResponsibleNamespacePrefix,Section FROM SetupAuditTrail ORDER BY CreatedDate ASC NULLS FIRST"] + responses: + - status_code: 200 + body: '{"done":true,"records":[{"Action":"insertConnectedApplication","CreatedByContext":"Einstein","CreatedById":"0055j000000utlPAAQ","CreatedByIssuer":null,"CreatedDate":"2022-08-16T09:26:38.000+0000","DelegateUser":"user1","Display":"For user user@elastic.co, the User Verified Email status changed to verified","Id":"0Ym5j000019nwonCAA","ResponsibleNamespacePrefix":"namespaceprefix","Section":"Connected Apps","attributes":{"type":"SetupAuditTrail","url":"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA"}}],"totalSize":160}' + headers: + content-type: ["text/json"] + - path: /services/data/v54.0/sobjects/EventLogFile/0AT5j00002GVrfnGAD/LogFile + methods: ["GET"] + responses: + - status_code: 200 + headers: + content-type: ["text/csv"] + body: |- + "EVENT_TYPE","TIMESTAMP","REQUEST_ID","ORGANIZATION_ID","USER_ID","RUN_TIME","CPU_TIME","URI","SESSION_KEY","LOGIN_KEY","USER_TYPE","REQUEST_STATUS","DB_TOTAL_TIME","BROWSER_TYPE","API_TYPE","API_VERSION","USER_NAME","TLS_PROTOCOL","CIPHER_SUITE","AUTHENTICATION_METHOD_REFERENCE","TIMESTAMP_DERIVED","USER_ID_DERIVED","CLIENT_IP","URI_ID_DERIVED","LOGIN_STATUS","SOURCE_IP" + "Login","20211006071307.550","4ehU_U-nbQyAPFl1cJILm-","00D5j000000VI3n","0055j000000utlP","83","30","/index.jsp","","QfNecrLXSII6fsBq","Standard","Success","52435102","Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36","f","9998.0","user@elastic.co","TLSv1.2","ECDHE-RSA-AES256-GCM-SHA384","","2021-10-06T07:13:07.550Z","0055j000000utlPAAQ","43.200.10.11","s4heK3WbH-lcJIL3-n","LOGIN_NO_ERROR","43.200.10.11" + - path: /services/data/v54.0/query + methods: ["GET"] + query_params: + q: ["SELECT Id,CreatedDate,LogDate,LogFile FROM EventLogFile WHERE Interval = 'Hourly' AND EventType = 'Logout' ORDER BY LogDate ASC NULLS FIRST"] + responses: + - status_code: 200 + body: '{"totalSize":1,"done":true,"records":[{"attributes":{"type":"EventLogFile","url":"/services/data/v54.0/sobjects/EventLogFile/0AT5j00002GWEsRGAX"},"Id":"0AT5j00002GWEsRGAX","CreatedDate":"2022-09-19T21:03:41.000+0000","LogDate":"2022-09-18T00:00:00.000+0000","LogFile":"/services/data/v54.0/sobjects/EventLogFile/0AT5j00002GWEsRGAX/LogFile"}]}' + headers: + content-type: ["text/json"] + - path: /services/data/v54.0/sobjects/EventLogFile/0AT5j00002GWEsRGAX/LogFile + methods: ["GET"] + responses: + - status_code: 200 + headers: + content-type: ["text/csv"] + body: |- + "EVENT_TYPE","TIMESTAMP","REQUEST_ID","ORGANIZATION_ID","USER_ID","USER_TYPE","SESSION_TYPE","SESSION_LEVEL","BROWSER_TYPE","PLATFORM_TYPE","RESOLUTION_TYPE","APP_TYPE","CLIENT_VERSION","API_TYPE","API_VERSION","USER_INITIATED_LOGOUT","SESSION_KEY","LOGIN_KEY","TIMESTAMP_DERIVED","USER_ID_DERIVED","CLIENT_IP" + "Logout","20220919073707.360","4exLFFQZ1234xFl1cJNwOV","00D5j001234VI3n","0055j000000utlP","S","O","1","Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36","1015","9999","1000","9998","f","54.0","0","WvtsJ1235oW24EbH","Obv9123BzbaxqCo1","2022-09-19T07:37:07.360Z","0055j000000utlPAAQ","43.200.10.11" diff --git a/packages/salesforce/changelog.yml b/packages/salesforce/changelog.yml new file mode 100644 index 00000000000..10057c13173 --- /dev/null +++ b/packages/salesforce/changelog.yml @@ -0,0 +1,7 @@ +# newer versions go on top + +- version: 0.1.0 + changes: + - description: Implement "setupaudittrail" data stream for Salesforce. + link: https://github.com/elastic/integrations/pull/1 + type: enhancement diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..c39dc386179 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,2 @@ +dynamic_fields: + event.ingested: ".*" diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log new file mode 100644 index 00000000000..a0d92d798a3 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log @@ -0,0 +1 @@ +{"Action":"insertConnectedApplication","CreatedByContext":"Einstein","CreatedById":"0055j000000utlPAAQ","CreatedByIssuer":null,"CreatedDate":"2022-08-16T09:26:38.000+0000","DelegateUser":"user1","Display":"For user user@elastic.co, the User Verified Email status changed to verified","Id":"0Ym5j000019nwonCAA","ResponsibleNamespacePrefix":"namespaceprefix","Section":"Connected Apps","attributes":{"type":"SetupAuditTrail","url":"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA"}} \ No newline at end of file diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json new file mode 100644 index 00000000000..25895ff37a4 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json @@ -0,0 +1,38 @@ +{ + "expected": [ + { + "@timestamp": "2022-08-16T09:26:38.000Z", + "ecs": { + "version": "8.4.0" + }, + "event": { + "action": "insertConnectedApplication", + "created": "2022-08-16T09:26:38.000Z", + "dataset": "salesforce.setupaudittrail", + "id": "0Ym5j000019nwonCAA", + "kind": "event", + "module": "salesforce", + "type": [ + "admin" + ], + "url": "/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA" + }, + "salesforce": { + "setup_audit_trail": { + "access_mode": "rest", + "created_by_context": "Einstein", + "created_by_id": "0055j000000utlPAAQ", + "delegate_user": "user1", + "display": "For user user@elastic.co, the User Verified Email status changed to verified", + "event_type": "SetupAuditTrail", + "responsible_namespace_prefix": "namespaceprefix", + "section": "Connected Apps" + } + }, + "user": { + "id": "0055j000000utlPAAQ", + "name": "user@elastic.co" + } + } + ] +} \ No newline at end of file diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml b/packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml new file mode 100644 index 00000000000..69c600d3e93 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml @@ -0,0 +1,12 @@ +input: httpjson +service: salesforce +vars: + instance_url: http://{{Hostname}}:{{Port}} + client_id: temp_client_id + client_secret: 40_characters_long_secret_key + username: temp_user + password: temp_password + token_url: http://{{Hostname}}:{{Port}}/services/oauth2/token +data_stream: + vars: + preserve_original_event: true diff --git a/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs new file mode 100644 index 00000000000..16da4f42873 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs @@ -0,0 +1,34 @@ +config_version: 2 +interval: {{period}} +request.method: GET +auth.oauth2: + enabled: true + client.id: {{client_id}} + client.secret: {{client_secret}} + token_url: {{token_url}} + user: {{username}} + password: {{password}} +request.url: {{instance_url}}/services/data/v54.0/query?q=setupaudittrail+rest +request.transforms: + - set: + target: url.params.q + value: "SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,ResponsibleNamespacePrefix,Section FROM SetupAuditTrail WHERE CreatedDate > [[.cursor.last_published_setupaudittrail]] ORDER BY CreatedDate ASC NULLS FIRST" + default: "SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,ResponsibleNamespacePrefix,Section FROM SetupAuditTrail ORDER BY CreatedDate ASC NULLS FIRST" +response.split: + target: body.records +cursor: + last_published_setupaudittrail: + value: '[[.last_event.CreatedDate]]' +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml b/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..a35f5f097f0 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,149 @@ +--- +description: Pipeline for parsing Salesforce SetupAuditTrail logs. +processors: +- json: + field: message + target_field: json +- rename: + field: message + target_field: event.original + ignore_missing: true + ignore_failure: true +- set: + field: salesforce.setup_audit_trail.access_mode + value: "rest" + ignore_failure: true + ignore_empty_value: true +- date: + field: json.CreatedDate + target_field: "@timestamp" + formats: + - ISO8601 + ignore_failure: true +- rename: + field: json.attributes.type + target_field: salesforce.setup_audit_trail.event_type + ignore_missing: true + ignore_failure: true +- rename: + field: json.CreatedByContext + target_field: salesforce.setup_audit_trail.created_by_context + ignore_missing: true + ignore_failure: true +- rename: + field: json.CreatedById + target_field: salesforce.setup_audit_trail.created_by_id + ignore_missing: true + ignore_failure: true +- rename: + field: json.CreatedByIssuer + target_field: salesforce.setup_audit_trail.created_by_issuer + ignore_missing: true + ignore_failure: true +- rename: + field: json.DelegateUser + target_field: salesforce.setup_audit_trail.delegate_user + ignore_missing: true + ignore_failure: true +- rename: + field: json.Display + target_field: salesforce.setup_audit_trail.display + ignore_missing: true + ignore_failure: true +- rename: + field: json.ResponsibleNamespacePrefix + target_field: salesforce.setup_audit_trail.responsible_namespace_prefix + ignore_missing: true + ignore_failure: true +- rename: + field: json.Section + target_field: salesforce.setup_audit_trail.section + ignore_missing: true + ignore_failure: true +- set: + field: ecs.version + value: 8.4.0 + ignore_failure: true + ignore_empty_value: true +- rename: + field: json.Id + target_field: event.id + ignore_missing: true + ignore_failure: true +- rename: + field: json.Action + target_field: event.action + ignore_missing: true + ignore_failure: true +- rename: + field: json.attributes.url + target_field: event.url + ignore_missing: true + ignore_failure: true +- date: + field: json.CreatedDate + target_field: event.created + formats: + - ISO8601 + ignore_failure: true +- set: + field: event.type + value: [admin] + ignore_failure: true + ignore_empty_value: true +- set: + field: event.kind + value: "event" + ignore_failure: true + ignore_empty_value: true +- set: + field: event.dataset + value: "salesforce.setupaudittrail" + ignore_failure: true + ignore_empty_value: true +- set: + field: event.module + value: "salesforce" + ignore_failure: true + ignore_empty_value: true +- set: + field: user.id + copy_from: salesforce.setup_audit_trail.created_by_id + ignore_failure: true + ignore_empty_value: true +- dissect: + field: salesforce.setup_audit_trail.display + pattern: "For user %{user.name}, %{?}" + ignore_failure: true +- script: + description: Drops null/empty values recursively. + lang: painless + source: | + boolean dropEmptyFields(Object object) { + if (object == null || object == "") { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(value -> dropEmptyFields(value)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(value -> dropEmptyFields(value)); + return (((List) object).length == 0); + } + return false; + } + dropEmptyFields(ctx); +- remove: + field: + - json + - message + ignore_missing: true + ignore_failure: true +- remove: + field: event.original + if: "ctx.tags == null || !(ctx.tags.contains('preserve_original_event'))" + ignore_failure: true + ignore_missing: true +on_failure: +- set: + field: error.message + value: '{{_ingest.on_failure_message}}' diff --git a/packages/salesforce/data_stream/setupaudittrail/fields/base-fields.yml b/packages/salesforce/data_stream/setupaudittrail/fields/base-fields.yml new file mode 100644 index 00000000000..3500f7ce8c2 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/fields/base-fields.yml @@ -0,0 +1,15 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: input.type + type: keyword + description: Input type. +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/salesforce/data_stream/setupaudittrail/fields/ecs.yml b/packages/salesforce/data_stream/setupaudittrail/fields/ecs.yml new file mode 100644 index 00000000000..180a757d8b7 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/fields/ecs.yml @@ -0,0 +1,28 @@ +- external: ecs + name: ecs.version +- external: ecs + name: error.message +- external: ecs + name: event.action +- external: ecs + name: event.created +- external: ecs + name: event.dataset +- external: ecs + name: event.id +- external: ecs + name: event.ingested +- external: ecs + name: event.kind +- external: ecs + name: event.module +- external: ecs + name: event.type +- external: ecs + name: event.url +- external: ecs + name: tags +- external: ecs + name: user.id +- external: ecs + name: user.name diff --git a/packages/salesforce/data_stream/setupaudittrail/fields/fields.yml b/packages/salesforce/data_stream/setupaudittrail/fields/fields.yml new file mode 100644 index 00000000000..d5da46c5dd7 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/fields/fields.yml @@ -0,0 +1,30 @@ +- name: salesforce.setup_audit_trail + type: group + fields: + - name: access_mode + type: keyword + description: Type of API from which the event is collected. + - name: created_by_context + type: keyword + description: The context under which the Setup change was made. For example, if Einstein uses cloud-to-cloud services to make a change in Setup, the value of this field is Einstein. + - name: created_by_id + type: keyword + description: Unknown. + - name: created_by_issuer + type: keyword + description: Reserved for future use. + - name: delegate_user + type: keyword + description: The Login-As user who executed the action in Setup. If a Login-As user didn’t perform the action, this field is blank. This field is available in API version 35.0 and later. + - name: display + type: keyword + description: "The full description of changes made in Setup. For example, if the Action field has a value of PermSetCreate, the Display field has a value like “Created permission set MAD: with user license Salesforce.”" + - name: event_type + type: keyword + description: Event type. + - name: responsible_namespace_prefix + type: keyword + description: Unknown. + - name: section + type: keyword + description: The section in the Setup menu where the action occurred. For example, Manage Users or Company Profile. diff --git a/packages/salesforce/data_stream/setupaudittrail/manifest.yml b/packages/salesforce/data_stream/setupaudittrail/manifest.yml new file mode 100644 index 00000000000..f010240fedf --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/manifest.yml @@ -0,0 +1,33 @@ +type: logs +title: Salesforce setupaudittrail logs +streams: + - input: httpjson + vars: + - name: period + type: text + title: Period + description: Period to query Salesforce. + multi: false + required: true + show_user: false + default: 1h + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - salesforce-setupaudittrail + - forwarded + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + template_path: httpjson.yml.hbs + title: Salesforce SetupAuditTrail logs + description: Collect Salesforce SetupAuditTrail logs. diff --git a/packages/salesforce/data_stream/setupaudittrail/sample_event.json b/packages/salesforce/data_stream/setupaudittrail/sample_event.json new file mode 100644 index 00000000000..70e04190ea1 --- /dev/null +++ b/packages/salesforce/data_stream/setupaudittrail/sample_event.json @@ -0,0 +1,62 @@ +{ + "@timestamp": "2022-08-16T09:26:38.000Z", + "agent": { + "ephemeral_id": "bcd0056b-54d8-4472-91ec-479be1e1d851", + "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.4.1" + }, + "data_stream": { + "dataset": "salesforce.setupaudittrail", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "snapshot": false, + "version": "8.4.1" + }, + "event": { + "action": "insertConnectedApplication", + "agent_id_status": "verified", + "created": "2022-08-16T09:26:38.000Z", + "dataset": "salesforce.setupaudittrail", + "id": "0Ym5j000019nwonCAA", + "ingested": "2022-09-28T10:07:25Z", + "kind": "event", + "module": "salesforce", + "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", + "type": [ + "admin" + ], + "url": "/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA" + }, + "input": { + "type": "httpjson" + }, + "salesforce": { + "setup_audit_trail": { + "access_mode": "rest", + "created_by_context": "Einstein", + "created_by_id": "0055j000000utlPAAQ", + "delegate_user": "user1", + "display": "For user user@elastic.co, the User Verified Email status changed to verified", + "event_type": "SetupAuditTrail", + "responsible_namespace_prefix": "namespaceprefix", + "section": "Connected Apps" + } + }, + "tags": [ + "preserve_original_event", + "salesforce-setupaudittrail", + "forwarded" + ], + "user": { + "id": "0055j000000utlPAAQ", + "name": "user@elastic.co" + } +} \ No newline at end of file diff --git a/packages/salesforce/docs/README.md b/packages/salesforce/docs/README.md new file mode 100644 index 00000000000..65dfd59ce15 --- /dev/null +++ b/packages/salesforce/docs/README.md @@ -0,0 +1,136 @@ +# Salesforce Integration + +## Overview + +The Salesforce integration allows you to monitor [Salesforce](https://www.salesforce.com/) instance. Salesforce provides customer relationship management service and also provides enterprise applications focused on customer service, marketing automation, analytics, and application development. + +Use the Salesforce integration to get visibility into the Salesforce Org operations and hold Salesforce accountable to the Service Level Agreements. Then visualize that data in Kibana, create alerts to notify you if something goes wrong, and reference logs when troubleshooting an issue. + +For example, if you want to check the number of successful and failed login attempts over time, you could check the same based on the ingested events or the visualization. Then you can create visualizations, alerts and troubleshoot by looking at the documents ingested in Elasticsearch. + +## Data streams + +The Salesforce integration collects log events using REST and Streaming API of Salesforce. + +**Logs** help you keep a record of events happening in Salesforce. +Log data streams collected by the Salesforce integration include [Login](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_login.htm) (using REST and Streaming API), [Logout](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_logout.htm) (using REST and Streaming API), [Apex](https://developer.salesforce.com/docs/atlas.en-us.238.0.object_reference.meta/object_reference/sforce_api_objects_apexclass.htm), and [SetupAuditTrail](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_setupaudittrail.htm). + +This integration uses: +- `httpjson` filebeat input to collect `login_rest`, `logout_rest`, `apex` and `setupaudittrail` events. +- `cometd` filebeat input to collect `login_stream` and `logout_stream` events. + +## Compatibility + +This integration has been tested against Salesforce API version `v54.0`. + +## Requirements + +You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. +You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware. + +## Logs reference + +### SetupAuditTrail + +This is the `setupaudittrail` data stream. It represents changes you or other admins made in your organization's Setup area for at least the last 180 days. + +An example event for `setupaudittrail` looks as following: + +```json +{ + "@timestamp": "2022-08-16T09:26:38.000Z", + "agent": { + "ephemeral_id": "bcd0056b-54d8-4472-91ec-479be1e1d851", + "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.4.1" + }, + "data_stream": { + "dataset": "salesforce.setupaudittrail", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "snapshot": false, + "version": "8.4.1" + }, + "event": { + "action": "insertConnectedApplication", + "agent_id_status": "verified", + "created": "2022-08-16T09:26:38.000Z", + "dataset": "salesforce.setupaudittrail", + "id": "0Ym5j000019nwonCAA", + "ingested": "2022-09-28T10:07:25Z", + "kind": "event", + "module": "salesforce", + "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", + "type": [ + "admin" + ], + "url": "/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA" + }, + "input": { + "type": "httpjson" + }, + "salesforce": { + "setup_audit_trail": { + "access_mode": "rest", + "created_by_context": "Einstein", + "created_by_id": "0055j000000utlPAAQ", + "delegate_user": "user1", + "display": "For user user@elastic.co, the User Verified Email status changed to verified", + "event_type": "SetupAuditTrail", + "responsible_namespace_prefix": "namespaceprefix", + "section": "Connected Apps" + } + }, + "tags": [ + "preserve_original_event", + "salesforce-setupaudittrail", + "forwarded" + ], + "user": { + "id": "0055j000000utlPAAQ", + "name": "user@elastic.co" + } +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword | +| error.message | Error message. | match_only_text | +| event.action | The action captured by the event. This describes the information in the event. It is more specific than `event.category`. Examples are `group-add`, `process-started`, `file-created`. The value is normally defined by the implementer. | keyword | +| event.created | event.created contains the date/time when the event was first read by an agent, or by your pipeline. This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event. In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. In case the two timestamps are identical, @timestamp should be used. | date | +| event.dataset | Name of the dataset. If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. | keyword | +| event.id | Unique ID to describe the event. | keyword | +| event.ingested | Timestamp when an event arrived in the central data store. This is different from `@timestamp`, which is when the event originally occurred. It's also different from `event.created`, which is meant to capture the first time an agent saw the event. In normal conditions, assuming no tampering, the timestamps should chronologically look like this: `@timestamp` \< `event.created` \< `event.ingested`. | date | +| event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not. | keyword | +| event.module | Name of the module this data is coming from. If your monitoring agent supports the concept of modules or plugins to process events of a given source (e.g. Apache logs), `event.module` should contain the name of this module. | keyword | +| event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. `event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword | +| event.url | URL linking to an external system to continue investigation of this event. This URL links to another system where in-depth investigation of the specific occurrence of this event can take place. Alert events, indicated by `event.kind:alert`, are a common use case for this field. | keyword | +| input.type | Input type. | keyword | +| salesforce.setup_audit_trail.access_mode | Type of API from which the event is collected. | keyword | +| salesforce.setup_audit_trail.created_by_context | The context under which the Setup change was made. For example, if Einstein uses cloud-to-cloud services to make a change in Setup, the value of this field is Einstein. | keyword | +| salesforce.setup_audit_trail.created_by_id | Unknown. | keyword | +| salesforce.setup_audit_trail.created_by_issuer | Reserved for future use. | keyword | +| salesforce.setup_audit_trail.delegate_user | The Login-As user who executed the action in Setup. If a Login-As user didn’t perform the action, this field is blank. This field is available in API version 35.0 and later. | keyword | +| salesforce.setup_audit_trail.display | The full description of changes made in Setup. For example, if the Action field has a value of PermSetCreate, the Display field has a value like “Created permission set MAD: with user license Salesforce.” | keyword | +| salesforce.setup_audit_trail.event_type | Event type. | keyword | +| salesforce.setup_audit_trail.responsible_namespace_prefix | Unknown. | keyword | +| salesforce.setup_audit_trail.section | The section in the Setup menu where the action occurred. For example, Manage Users or Company Profile. | keyword | +| tags | List of keywords used to tag each event. | keyword | +| user.id | Unique identifier of the user. | keyword | +| user.name | Short name or login of the user. | keyword | +| user.name.text | Multi-field of `user.name`. | match_only_text | diff --git a/packages/salesforce/img/salesforce.svg b/packages/salesforce/img/salesforce.svg new file mode 100644 index 00000000000..4aaaf3ca976 --- /dev/null +++ b/packages/salesforce/img/salesforce.svg @@ -0,0 +1,25 @@ + + +Salesforce.com logo +A cloud computing company based in San Francisco, California, United States + + + + image/svg+xml + + + + + + + + + + + + + + + + + diff --git a/packages/salesforce/manifest.yml b/packages/salesforce/manifest.yml new file mode 100644 index 00000000000..7112c0579cf --- /dev/null +++ b/packages/salesforce/manifest.yml @@ -0,0 +1,66 @@ +format_version: 1.0.0 +name: salesforce +title: Salesforce +version: 0.1.0 +license: basic +description: Collect logs from Salesforce with Elastic Agent. +type: integration +categories: + - crm +conditions: + kibana.version: ^8.4.0 +icons: + - src: /img/salesforce.svg + title: Salesforce + size: 32x32 + type: image/svg+xml +policy_templates: + - name: salesforce + title: Salesforce logs + description: Collect logs from Salesforce instances. + inputs: + - type: httpjson + vars: + - name: instance_url + type: text + title: Instance URL + description: The Instance URL of the Salesforce instance. + required: true + show_user: true + default: https://instance-url.salesforce.com + - name: client_id + type: text + title: Client ID + description: OAuth 2.0 client ID. + required: true + show_user: true + - name: client_secret + type: password + title: Client Secret + description: OAuth 2.0 client secret. + required: true + show_user: true + - name: username + type: text + title: Username + description: The username used as part of the authentication flow. + required: true + show_user: true + default: my.email@here.com + - name: password + type: password + title: Password + description: The password used as part of the authentication flow. + required: true + show_user: true + - name: token_url + type: text + title: Token URL + description: The OAuth 2.0 token URL for Salesforce. + required: true + show_user: false + default: https://login.salesforce.com/services/oauth2/token + title: Collect Salesforce logs using REST API + description: Collecting logs using REST API from Salesforce instances. +owner: + github: elastic/obs-service-integrations From eed72a259f6842878125717fb8afe46ca13d9479 Mon Sep 17 00:00:00 2001 From: yug-elastic Date: Sat, 1 Oct 2022 21:38:19 +0530 Subject: [PATCH 2/7] Add changelog entry --- packages/salesforce/changelog.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/salesforce/changelog.yml b/packages/salesforce/changelog.yml index 10057c13173..178b6dc0107 100644 --- a/packages/salesforce/changelog.yml +++ b/packages/salesforce/changelog.yml @@ -3,5 +3,5 @@ - version: 0.1.0 changes: - description: Implement "setupaudittrail" data stream for Salesforce. - link: https://github.com/elastic/integrations/pull/1 + link: https://github.com/elastic/integrations/pull/4356 type: enhancement From cef39e9f5dd4a184564abbfb91db18d044fcf8d8 Mon Sep 17 00:00:00 2001 From: yug-elastic Date: Fri, 7 Oct 2022 12:17:49 +0530 Subject: [PATCH 3/7] Minor updates to the pipeline --- .../_dev/deploy/docker/files/config.yml | 18 ++++++++++++++++++ .../_dev/test/system/test-default-config.yml | 2 +- .../elasticsearch/ingest_pipeline/default.yml | 7 ++++--- .../setupaudittrail/sample_event.json | 8 ++++---- packages/salesforce/docs/README.md | 8 ++++---- 5 files changed, 31 insertions(+), 12 deletions(-) diff --git a/packages/salesforce/_dev/deploy/docker/files/config.yml b/packages/salesforce/_dev/deploy/docker/files/config.yml index c623cfbafa1..d0bb85fd35a 100644 --- a/packages/salesforce/_dev/deploy/docker/files/config.yml +++ b/packages/salesforce/_dev/deploy/docker/files/config.yml @@ -51,3 +51,21 @@ rules: body: |- "EVENT_TYPE","TIMESTAMP","REQUEST_ID","ORGANIZATION_ID","USER_ID","USER_TYPE","SESSION_TYPE","SESSION_LEVEL","BROWSER_TYPE","PLATFORM_TYPE","RESOLUTION_TYPE","APP_TYPE","CLIENT_VERSION","API_TYPE","API_VERSION","USER_INITIATED_LOGOUT","SESSION_KEY","LOGIN_KEY","TIMESTAMP_DERIVED","USER_ID_DERIVED","CLIENT_IP" "Logout","20220919073707.360","4exLFFQZ1234xFl1cJNwOV","00D5j001234VI3n","0055j000000utlP","S","O","1","Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36","1015","9999","1000","9998","f","54.0","0","WvtsJ1235oW24EbH","Obv9123BzbaxqCo1","2022-09-19T07:37:07.360Z","0055j000000utlPAAQ","43.200.10.11" + - path: /services/data/v54.0/query + methods: ["GET"] + query_params: + q: ["SELECT Id,CreatedDate,LogDate,LogFile FROM EventLogFile WHERE Interval = 'Hourly' AND (EventType = 'ApexCallout' OR EventType = 'ApexExecution' OR EventType = 'ApexRestApi' OR EventType = 'ApexSoap' OR EventType = 'ApexTrigger' OR EventType = 'ExternalCustomApexCallout') ORDER BY LogDate ASC NULLS FIRST"] + responses: + - status_code: 200 + body: '{"done":true,"records":[{"CreatedDate":"2022-10-01T23:22:27.000+0000","Id":"0AT5j00002GWEsRGAY","LogDate":"2022-09-30T00:00:00.000+0000","LogFile":"/services/data/v54.0/sobjects/EventLogFile/0AT5j00002GWEsRGAY/LogFile","attributes":{"type":"EventLogFile","url":"/services/data/v54.0/sobjects/EventLogFile/0AT5j00002GWEsRGAY"}}],"totalSize":1}' + headers: + content-type: ["text/json"] + - path: /services/data/v54.0/sobjects/EventLogFile/0AT5j00002GWEsRGAY/LogFile + methods: ["GET"] + responses: + - status_code: 200 + headers: + content-type: ["text/csv"] + body: |- + "EVENT_TYPE","TIMESTAMP","REQUEST_ID","ORGANIZATION_ID","USER_ID","RUN_TIME","CPU_TIME","URI","SESSION_KEY","LOGIN_KEY","TYPE","METHOD","SUCCESS","TIME","REQUEST_SIZE","RESPONSE_SIZE","URL","TIMESTAMP_DERIVED","USER_ID_DERIVED","CLIENT_IP","URI_ID_DERIVED" + "ApexCallout","20211102091250.488","ABCDE","00D5j000000VABC","0055j000000ABCD","1305","10","CALLOUT-LOG","ABCDEF","ABCDEFGH","OData","GET","1","1293","10","256","https://temp.sh/odata/Accounts","2021-11-02T09:12:50.488Z","0055j012345utlPAAQ","127.0.0.1","0055j000000utlPABCD" diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml b/packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml index 69c600d3e93..c6aa6813ff0 100644 --- a/packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/system/test-default-config.yml @@ -3,7 +3,7 @@ service: salesforce vars: instance_url: http://{{Hostname}}:{{Port}} client_id: temp_client_id - client_secret: 40_characters_long_secret_key + client_secret: forty_characters_long_secret_key username: temp_user password: temp_password token_url: http://{{Hostname}}:{{Port}}/services/oauth2/token diff --git a/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml b/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml index a35f5f097f0..8f77379fd83 100644 --- a/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml +++ b/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml @@ -4,6 +4,7 @@ processors: - json: field: message target_field: json + ignore_failure: true - rename: field: message target_field: event.original @@ -62,7 +63,7 @@ processors: ignore_failure: true - set: field: ecs.version - value: 8.4.0 + value: "8.4.0" ignore_failure: true ignore_empty_value: true - rename: @@ -88,7 +89,7 @@ processors: ignore_failure: true - set: field: event.type - value: [admin] + value: ["admin"] ignore_failure: true ignore_empty_value: true - set: @@ -146,4 +147,4 @@ processors: on_failure: - set: field: error.message - value: '{{_ingest.on_failure_message}}' + value: '{{{_ingest.on_failure_message}}}' diff --git a/packages/salesforce/data_stream/setupaudittrail/sample_event.json b/packages/salesforce/data_stream/setupaudittrail/sample_event.json index 70e04190ea1..cdbaec82fe8 100644 --- a/packages/salesforce/data_stream/setupaudittrail/sample_event.json +++ b/packages/salesforce/data_stream/setupaudittrail/sample_event.json @@ -1,8 +1,8 @@ { "@timestamp": "2022-08-16T09:26:38.000Z", "agent": { - "ephemeral_id": "bcd0056b-54d8-4472-91ec-479be1e1d851", - "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "ephemeral_id": "f3d13e01-7cfb-4e01-9bbd-99e08c5157b1", + "id": "dbe82fcc-9eea-4080-91fe-9f4a6afa87ee", "name": "docker-fleet-agent", "type": "filebeat", "version": "8.4.1" @@ -16,7 +16,7 @@ "version": "8.4.0" }, "elastic_agent": { - "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "id": "dbe82fcc-9eea-4080-91fe-9f4a6afa87ee", "snapshot": false, "version": "8.4.1" }, @@ -26,7 +26,7 @@ "created": "2022-08-16T09:26:38.000Z", "dataset": "salesforce.setupaudittrail", "id": "0Ym5j000019nwonCAA", - "ingested": "2022-09-28T10:07:25Z", + "ingested": "2022-10-04T11:43:33Z", "kind": "event", "module": "salesforce", "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", diff --git a/packages/salesforce/docs/README.md b/packages/salesforce/docs/README.md index 65dfd59ce15..9dae2e0f8a2 100644 --- a/packages/salesforce/docs/README.md +++ b/packages/salesforce/docs/README.md @@ -40,8 +40,8 @@ An example event for `setupaudittrail` looks as following: { "@timestamp": "2022-08-16T09:26:38.000Z", "agent": { - "ephemeral_id": "bcd0056b-54d8-4472-91ec-479be1e1d851", - "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "ephemeral_id": "f3d13e01-7cfb-4e01-9bbd-99e08c5157b1", + "id": "dbe82fcc-9eea-4080-91fe-9f4a6afa87ee", "name": "docker-fleet-agent", "type": "filebeat", "version": "8.4.1" @@ -55,7 +55,7 @@ An example event for `setupaudittrail` looks as following: "version": "8.4.0" }, "elastic_agent": { - "id": "74e83e88-b0f6-4eaa-ab9c-ccf00abcf412", + "id": "dbe82fcc-9eea-4080-91fe-9f4a6afa87ee", "snapshot": false, "version": "8.4.1" }, @@ -65,7 +65,7 @@ An example event for `setupaudittrail` looks as following: "created": "2022-08-16T09:26:38.000Z", "dataset": "salesforce.setupaudittrail", "id": "0Ym5j000019nwonCAA", - "ingested": "2022-09-28T10:07:25Z", + "ingested": "2022-10-04T11:43:33Z", "kind": "event", "module": "salesforce", "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", From 9b381bab58ef16f0f0cf9dc691d5593697e04da9 Mon Sep 17 00:00:00 2001 From: yug-elastic Date: Mon, 17 Oct 2022 19:18:35 +0530 Subject: [PATCH 4/7] Update README as per review comments, other minor changes --- packages/salesforce/_dev/build/docs/README.md | 82 ++++++++++++++++++- .../_dev/test/pipeline/test-common-config.yml | 3 + .../test-setupaudittrail.log-expected.json | 4 + .../agent/stream/httpjson.yml.hbs | 1 + .../data_stream/setupaudittrail/manifest.yml | 9 +- packages/salesforce/docs/README.md | 82 ++++++++++++++++++- 6 files changed, 172 insertions(+), 9 deletions(-) diff --git a/packages/salesforce/_dev/build/docs/README.md b/packages/salesforce/_dev/build/docs/README.md index b3ba0ee0bb5..754faed8099 100644 --- a/packages/salesforce/_dev/build/docs/README.md +++ b/packages/salesforce/_dev/build/docs/README.md @@ -15,19 +15,93 @@ The Salesforce integration collects log events using REST and Streaming API of S **Logs** help you keep a record of events happening in Salesforce. Log data streams collected by the Salesforce integration include [Login](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_login.htm) (using REST and Streaming API), [Logout](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_logout.htm) (using REST and Streaming API), [Apex](https://developer.salesforce.com/docs/atlas.en-us.238.0.object_reference.meta/object_reference/sforce_api_objects_apexclass.htm), and [SetupAuditTrail](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_setupaudittrail.htm). -This integration uses: -- `httpjson` filebeat input to collect `login_rest`, `logout_rest`, `apex` and `setupaudittrail` events. -- `cometd` filebeat input to collect `login_stream` and `logout_stream` events. +Data stream names: +- `login_rest` +- `logout_rest` +- `apex` +- `setupaudittrail` +- `login_stream` +- `logout_stream` ## Compatibility -This integration has been tested against Salesforce API version `v54.0`. +This integration has been tested against Salesforce `Spring '22 (v54.0) release`. ## Requirements You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware. +### Steps to find out the version of Salesforce + +On the Home tab in Salesforce Classic, on the top right of the screen is a link to release like `Summer '22`. This indicates your release. + +Another alternative to find out the version of Salesforce is hitting the following URL: +- Format: (Salesforce Instance URL)/services/data +- Example: https://elastic1234-dev-ed.my.salesforce.com/services/data + +Example response: +```xml + + + + /services/data/v53.0 + 53.0 + + + + /services/data/v54.0 + 54.0 + + + + /services/data/v55.0 + 55.0 + + +``` +The last one in the list is the release of your instance. In the example above, the version is `Summer '22` i.e. `v55.0`. + +### Steps to find out the API version + +- Go to `Setup` > `Quick Find` > `Apex` > `Apex Classes` +- Then, click `New` button +- Then, click over to `Version Settings` tab +- Reference the `Version` drop down for the API Version number. + +### Prerequisite + +In order to use this integration, you will need to create a new SFDC Application using OAuth. More details can be found [here](https://help.salesforce.com/apex/HTViewHelpDoc?id=connected_app_create.htm). + +Create a Connected App in Salesforce: + +1. Login to [Salesforce](https://login.salesforce.com/) with the same user credentials that you want to collect data. + +2. From Setup, enter "App Manager" in the Quick Find box, then select "App Manager". + +3. Click *New Connected App*. + +4. Enter the connected app's name, which displays in the App Manager and on its App Launcher tile. + +5. Enter the API name. The default is a version of the name without spaces. Only letters, numbers, and underscores are allowed. If the original app name contains any other characters, edit the default name. + +6. Enter the contact email for Salesforce. + +7. In the API (Enable OAuth Settings) area of the page, select *Enable OAuth Settings*. + +8. Select the following OAuth scopes to apply to the connected app: +- Access and manage your data (API). +- Perform requests on your behalf at any time (refresh_token, offline_access). +- (Optional) In case of data collection, if any permission issues arise, add the Full access (full) scope. + +9. Select *Require Secret for the Web Server Flow* to require the app's client secret in exchange for an access token. + +10. Select *Require Secret for Refresh Token Flow* to require the app's client secret in the authorization request of a refresh token and hybrid refresh token flow. + +11. Click Save. It can take about 10 minutes for the changes to take effect. + +12. Take Consumer Key and Secret from the Connected App API section. + ## Logs reference ### SetupAuditTrail diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml index c39dc386179..5622947e4b8 100644 --- a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-common-config.yml @@ -1,2 +1,5 @@ dynamic_fields: event.ingested: ".*" +fields: + tags: + - preserve_original_event diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json index 25895ff37a4..efb1780610f 100644 --- a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json @@ -12,6 +12,7 @@ "id": "0Ym5j000019nwonCAA", "kind": "event", "module": "salesforce", + "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", "type": [ "admin" ], @@ -29,6 +30,9 @@ "section": "Connected Apps" } }, + "tags": [ + "preserve_original_event" + ], "user": { "id": "0055j000000utlPAAQ", "name": "user@elastic.co" diff --git a/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs index 16da4f42873..0d2f8dd84b7 100644 --- a/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs +++ b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs @@ -30,5 +30,6 @@ tags: publisher_pipeline.disable_host: true {{/contains}} {{#if processors}} +processors: {{processors}} {{/if}} diff --git a/packages/salesforce/data_stream/setupaudittrail/manifest.yml b/packages/salesforce/data_stream/setupaudittrail/manifest.yml index f010240fedf..12997f71688 100644 --- a/packages/salesforce/data_stream/setupaudittrail/manifest.yml +++ b/packages/salesforce/data_stream/setupaudittrail/manifest.yml @@ -6,7 +6,7 @@ streams: - name: period type: text title: Period - description: Period to query Salesforce. + description: Period of fetching metrics, i.e. 1s/1m/1h. multi: false required: true show_user: false @@ -28,6 +28,13 @@ streams: type: bool multi: false default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. template_path: httpjson.yml.hbs title: Salesforce SetupAuditTrail logs description: Collect Salesforce SetupAuditTrail logs. diff --git a/packages/salesforce/docs/README.md b/packages/salesforce/docs/README.md index 9dae2e0f8a2..0ed991505a8 100644 --- a/packages/salesforce/docs/README.md +++ b/packages/salesforce/docs/README.md @@ -15,19 +15,93 @@ The Salesforce integration collects log events using REST and Streaming API of S **Logs** help you keep a record of events happening in Salesforce. Log data streams collected by the Salesforce integration include [Login](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_login.htm) (using REST and Streaming API), [Logout](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_logout.htm) (using REST and Streaming API), [Apex](https://developer.salesforce.com/docs/atlas.en-us.238.0.object_reference.meta/object_reference/sforce_api_objects_apexclass.htm), and [SetupAuditTrail](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_setupaudittrail.htm). -This integration uses: -- `httpjson` filebeat input to collect `login_rest`, `logout_rest`, `apex` and `setupaudittrail` events. -- `cometd` filebeat input to collect `login_stream` and `logout_stream` events. +Data stream names: +- `login_rest` +- `logout_rest` +- `apex` +- `setupaudittrail` +- `login_stream` +- `logout_stream` ## Compatibility -This integration has been tested against Salesforce API version `v54.0`. +This integration has been tested against Salesforce `Spring '22 (v54.0) release`. ## Requirements You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware. +### Steps to find out the version of Salesforce + +On the Home tab in Salesforce Classic, on the top right of the screen is a link to release like `Summer '22`. This indicates your release. + +Another alternative to find out the version of Salesforce is hitting the following URL: +- Format: (Salesforce Instance URL)/services/data +- Example: https://elastic1234-dev-ed.my.salesforce.com/services/data + +Example response: +```xml + + + + /services/data/v53.0 + 53.0 + + + + /services/data/v54.0 + 54.0 + + + + /services/data/v55.0 + 55.0 + + +``` +The last one in the list is the release of your instance. In the example above, the version is `Summer '22` i.e. `v55.0`. + +### Steps to find out the API version + +- Go to `Setup` > `Quick Find` > `Apex` > `Apex Classes` +- Then, click `New` button +- Then, click over to `Version Settings` tab +- Reference the `Version` drop down for the API Version number. + +### Prerequisite + +In order to use this integration, you will need to create a new SFDC Application using OAuth. More details can be found [here](https://help.salesforce.com/apex/HTViewHelpDoc?id=connected_app_create.htm). + +Create a Connected App in Salesforce: + +1. Login to [Salesforce](https://login.salesforce.com/) with the same user credentials that you want to collect data. + +2. From Setup, enter "App Manager" in the Quick Find box, then select "App Manager". + +3. Click *New Connected App*. + +4. Enter the connected app's name, which displays in the App Manager and on its App Launcher tile. + +5. Enter the API name. The default is a version of the name without spaces. Only letters, numbers, and underscores are allowed. If the original app name contains any other characters, edit the default name. + +6. Enter the contact email for Salesforce. + +7. In the API (Enable OAuth Settings) area of the page, select *Enable OAuth Settings*. + +8. Select the following OAuth scopes to apply to the connected app: +- Access and manage your data (API). +- Perform requests on your behalf at any time (refresh_token, offline_access). +- (Optional) In case of data collection, if any permission issues arise, add the Full access (full) scope. + +9. Select *Require Secret for the Web Server Flow* to require the app's client secret in exchange for an access token. + +10. Select *Require Secret for Refresh Token Flow* to require the app's client secret in the authorization request of a refresh token and hybrid refresh token flow. + +11. Click Save. It can take about 10 minutes for the changes to take effect. + +12. Take Consumer Key and Secret from the Connected App API section. + ## Logs reference ### SetupAuditTrail From d96a8bc82a9db26260e39ede1940b11eabc096e3 Mon Sep 17 00:00:00 2001 From: kush-elastic Date: Mon, 19 Dec 2022 21:27:46 +0530 Subject: [PATCH 5/7] Add pagination support --- .../setupaudittrail/agent/stream/httpjson.yml.hbs | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs index 77da985a6de..2d75af8481e 100644 --- a/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs +++ b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs @@ -16,6 +16,11 @@ request.transforms: default: "SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,ResponsibleNamespacePrefix,Section FROM SetupAuditTrail ORDER BY CreatedDate ASC NULLS FIRST" response.split: target: body.records +response.pagination: +- set: + target: url.value + value: '[[if (ne .last_response.body.done true)]]{{instance_url}}[[.last_response.body.nextRecordsUrl]][[end]]' + fail_on_template_error: true cursor: last_published_setupaudittrail: value: '[[.last_event.CreatedDate]]' From cb0ec8f164adf3327a6c42ccd2f1415bdf7a36cc Mon Sep 17 00:00:00 2001 From: kush-elastic Date: Wed, 4 Jan 2023 21:38:12 +0530 Subject: [PATCH 6/7] Address review comments --- .../_dev/deploy/docker/files/config.yml | 4 ++-- .../test/pipeline/test-setupaudittrail.log | 2 +- .../test-setupaudittrail.log-expected.json | 3 +-- .../agent/stream/httpjson.yml.hbs | 4 ++-- .../elasticsearch/ingest_pipeline/default.yml | 5 ----- .../setupaudittrail/fields/fields.yml | 9 +++------ .../setupaudittrail/sample_event.json | 11 +++++------ packages/salesforce/docs/README.md | 18 ++++++++---------- 8 files changed, 22 insertions(+), 34 deletions(-) diff --git a/packages/salesforce/_dev/deploy/docker/files/config.yml b/packages/salesforce/_dev/deploy/docker/files/config.yml index 354a307bc76..461757fd10a 100644 --- a/packages/salesforce/_dev/deploy/docker/files/config.yml +++ b/packages/salesforce/_dev/deploy/docker/files/config.yml @@ -18,10 +18,10 @@ rules: - path: /services/data/v54.0/query methods: ["GET"] query_params: - q: ["SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,ResponsibleNamespacePrefix,Section FROM SetupAuditTrail ORDER BY CreatedDate ASC NULLS FIRST"] + q: ["SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,Section FROM SetupAuditTrail ORDER BY CreatedDate ASC NULLS FIRST"] responses: - status_code: 200 - body: '{"done":true,"records":[{"Action":"insertConnectedApplication","CreatedByContext":"Einstein","CreatedById":"0055j000000utlPAAQ","CreatedByIssuer":null,"CreatedDate":"2022-08-16T09:26:38.000+0000","DelegateUser":"user1","Display":"For user user@elastic.co, the User Verified Email status changed to verified","Id":"0Ym5j000019nwonCAA","ResponsibleNamespacePrefix":"namespaceprefix","Section":"Connected Apps","attributes":{"type":"SetupAuditTrail","url":"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA"}}],"totalSize":160}' + body: '{"done":true,"records":[{"Action":"insertConnectedApplication","CreatedByContext":"Einstein","CreatedById":"0055j000000utlPAAQ","CreatedByIssuer":null,"CreatedDate":"2022-08-16T09:26:38.000+0000","DelegateUser":"user1","Display":"For user user@elastic.co, the User Verified Email status changed to verified","Id":"0Ym5j000019nwonCAA","Section":"Connected Apps","attributes":{"type":"SetupAuditTrail","url":"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA"}}],"totalSize":160}' headers: content-type: ["text/json"] - path: /services/data/v54.0/sobjects/EventLogFile/0AT5j00002GVrfnGAD/LogFile diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log index a0d92d798a3..8021e715dff 100644 --- a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log @@ -1 +1 @@ -{"Action":"insertConnectedApplication","CreatedByContext":"Einstein","CreatedById":"0055j000000utlPAAQ","CreatedByIssuer":null,"CreatedDate":"2022-08-16T09:26:38.000+0000","DelegateUser":"user1","Display":"For user user@elastic.co, the User Verified Email status changed to verified","Id":"0Ym5j000019nwonCAA","ResponsibleNamespacePrefix":"namespaceprefix","Section":"Connected Apps","attributes":{"type":"SetupAuditTrail","url":"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA"}} \ No newline at end of file +{"Action":"insertConnectedApplication","CreatedByContext":"Einstein","CreatedById":"0055j000000utlPAAQ","CreatedByIssuer":null,"CreatedDate":"2022-08-16T09:26:38.000+0000","DelegateUser":"user1","Display":"For user user@elastic.co, the User Verified Email status changed to verified","Id":"0Ym5j000019nwonCAA","Section":"Connected Apps","attributes":{"type":"SetupAuditTrail","url":"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA"}} \ No newline at end of file diff --git a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json index 894aaa57890..ae92f30bccd 100644 --- a/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json +++ b/packages/salesforce/data_stream/setupaudittrail/_dev/test/pipeline/test-setupaudittrail.log-expected.json @@ -12,7 +12,7 @@ "id": "0Ym5j000019nwonCAA", "kind": "event", "module": "salesforce", - "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", + "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", "type": [ "admin" ], @@ -26,7 +26,6 @@ "delegate_user": "user1", "display": "For user user@elastic.co, the User Verified Email status changed to verified", "event_type": "SetupAuditTrail", - "responsible_namespace_prefix": "namespaceprefix", "section": "Connected Apps" } }, diff --git a/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs index 2d75af8481e..d3cebb0412e 100644 --- a/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs +++ b/packages/salesforce/data_stream/setupaudittrail/agent/stream/httpjson.yml.hbs @@ -12,8 +12,8 @@ request.url: {{instance_url}}/services/data/v54.0/query?q=setupaudittrail+rest request.transforms: - set: target: url.params.q - value: "SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,ResponsibleNamespacePrefix,Section FROM SetupAuditTrail WHERE CreatedDate > [[.cursor.last_published_setupaudittrail]] ORDER BY CreatedDate ASC NULLS FIRST" - default: "SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,ResponsibleNamespacePrefix,Section FROM SetupAuditTrail ORDER BY CreatedDate ASC NULLS FIRST" + value: "SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,Section FROM SetupAuditTrail WHERE CreatedDate > [[.cursor.last_published_setupaudittrail]] ORDER BY CreatedDate ASC NULLS FIRST" + default: "SELECT Action,CreatedByContext,CreatedById,CreatedByIssuer,CreatedDate,DelegateUser,Display,Id,Section FROM SetupAuditTrail ORDER BY CreatedDate ASC NULLS FIRST" response.split: target: body.records response.pagination: diff --git a/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml b/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml index 0b695deda1a..9fb9c6f8458 100644 --- a/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml +++ b/packages/salesforce/data_stream/setupaudittrail/elasticsearch/ingest_pipeline/default.yml @@ -51,11 +51,6 @@ processors: target_field: salesforce.setup_audit_trail.display ignore_missing: true ignore_failure: true -- rename: - field: json.ResponsibleNamespacePrefix - target_field: salesforce.setup_audit_trail.responsible_namespace_prefix - ignore_missing: true - ignore_failure: true - rename: field: json.Section target_field: salesforce.setup_audit_trail.section diff --git a/packages/salesforce/data_stream/setupaudittrail/fields/fields.yml b/packages/salesforce/data_stream/setupaudittrail/fields/fields.yml index 49ec712c7a6..e540673a155 100644 --- a/packages/salesforce/data_stream/setupaudittrail/fields/fields.yml +++ b/packages/salesforce/data_stream/setupaudittrail/fields/fields.yml @@ -15,22 +15,19 @@ description: The context under which the Setup change was made. For example, if Einstein uses cloud-to-cloud services to make a change in Setup, the value of this field is Einstein. - name: created_by_id type: keyword - description: Unknown. + description: The id under which the Setup change was made. For example, if Einstein uses cloud-to-cloud services to make a change in Setup, the value of this field is id of Einstein. - name: created_by_issuer type: keyword description: Reserved for future use. - name: delegate_user type: keyword - description: The Login-As user who executed the action in Setup. If a Login-As user didn’t perform the action, this field is blank. This field is available in API version 35.0 and later. + description: The Login-As user who executed the action in Setup. If a Login-As user didn't perform the action, this field is empty string. This field is available in API version 35.0 and later. - name: display type: keyword - description: "The full description of changes made in Setup. For example, if the Action field has a value of PermSetCreate, the Display field has a value like “Created permission set MAD: with user license Salesforce.”" + description: 'The full description of changes made in Setup. For example, if the event.action field has a value of PermSetCreate, the Display field has a value like "Created permission set MAD: with user license Salesforce."' - name: event_type type: keyword description: Event type. - - name: responsible_namespace_prefix - type: keyword - description: Unknown. - name: section type: keyword description: The section in the Setup menu where the action occurred. For example, Manage Users or Company Profile. diff --git a/packages/salesforce/data_stream/setupaudittrail/sample_event.json b/packages/salesforce/data_stream/setupaudittrail/sample_event.json index a1fcd15de2c..0d75af9cddf 100644 --- a/packages/salesforce/data_stream/setupaudittrail/sample_event.json +++ b/packages/salesforce/data_stream/setupaudittrail/sample_event.json @@ -1,8 +1,8 @@ { "@timestamp": "2022-08-16T09:26:38.000Z", "agent": { - "ephemeral_id": "27d615c9-0738-49b3-9593-c41aeb5c2e0c", - "id": "e8ad8355-f296-4e32-9096-2df7c9cc7e97", + "ephemeral_id": "cf463665-f17d-4155-8434-4f93e0fabd18", + "id": "511d10d2-be41-45d0-9712-40b7ce035864", "name": "docker-fleet-agent", "type": "filebeat", "version": "8.4.1" @@ -16,7 +16,7 @@ "version": "8.5.0" }, "elastic_agent": { - "id": "e8ad8355-f296-4e32-9096-2df7c9cc7e97", + "id": "511d10d2-be41-45d0-9712-40b7ce035864", "snapshot": false, "version": "8.4.1" }, @@ -26,10 +26,10 @@ "created": "2022-08-16T09:26:38.000Z", "dataset": "salesforce.setupaudittrail", "id": "0Ym5j000019nwonCAA", - "ingested": "2022-12-15T10:30:36Z", + "ingested": "2023-01-04T15:34:45Z", "kind": "event", "module": "salesforce", - "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", + "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", "type": [ "admin" ], @@ -47,7 +47,6 @@ "delegate_user": "user1", "display": "For user user@elastic.co, the User Verified Email status changed to verified", "event_type": "SetupAuditTrail", - "responsible_namespace_prefix": "namespaceprefix", "section": "Connected Apps" } }, diff --git a/packages/salesforce/docs/README.md b/packages/salesforce/docs/README.md index e4517ac5c91..56d16831c0d 100644 --- a/packages/salesforce/docs/README.md +++ b/packages/salesforce/docs/README.md @@ -499,8 +499,8 @@ An example event for `setupaudittrail` looks as following: { "@timestamp": "2022-08-16T09:26:38.000Z", "agent": { - "ephemeral_id": "27d615c9-0738-49b3-9593-c41aeb5c2e0c", - "id": "e8ad8355-f296-4e32-9096-2df7c9cc7e97", + "ephemeral_id": "cf463665-f17d-4155-8434-4f93e0fabd18", + "id": "511d10d2-be41-45d0-9712-40b7ce035864", "name": "docker-fleet-agent", "type": "filebeat", "version": "8.4.1" @@ -514,7 +514,7 @@ An example event for `setupaudittrail` looks as following: "version": "8.5.0" }, "elastic_agent": { - "id": "e8ad8355-f296-4e32-9096-2df7c9cc7e97", + "id": "511d10d2-be41-45d0-9712-40b7ce035864", "snapshot": false, "version": "8.4.1" }, @@ -524,10 +524,10 @@ An example event for `setupaudittrail` looks as following: "created": "2022-08-16T09:26:38.000Z", "dataset": "salesforce.setupaudittrail", "id": "0Ym5j000019nwonCAA", - "ingested": "2022-12-15T10:30:36Z", + "ingested": "2023-01-04T15:34:45Z", "kind": "event", "module": "salesforce", - "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"ResponsibleNamespacePrefix\":\"namespaceprefix\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", + "original": "{\"Action\":\"insertConnectedApplication\",\"CreatedByContext\":\"Einstein\",\"CreatedById\":\"0055j000000utlPAAQ\",\"CreatedByIssuer\":null,\"CreatedDate\":\"2022-08-16T09:26:38.000+0000\",\"DelegateUser\":\"user1\",\"Display\":\"For user user@elastic.co, the User Verified Email status changed to verified\",\"Id\":\"0Ym5j000019nwonCAA\",\"Section\":\"Connected Apps\",\"attributes\":{\"type\":\"SetupAuditTrail\",\"url\":\"/services/data/v54.0/sobjects/SetupAuditTrail/0Ym5j000019nwonCAA\"}}", "type": [ "admin" ], @@ -545,7 +545,6 @@ An example event for `setupaudittrail` looks as following: "delegate_user": "user1", "display": "For user user@elastic.co, the User Verified Email status changed to verified", "event_type": "SetupAuditTrail", - "responsible_namespace_prefix": "namespaceprefix", "section": "Connected Apps" } }, @@ -584,12 +583,11 @@ An example event for `setupaudittrail` looks as following: | salesforce.instance_url | The Instance URL of the Salesforce instance. | keyword | | salesforce.setup_audit_trail.access_mode | Type of API from which the event is collected. | keyword | | salesforce.setup_audit_trail.created_by_context | The context under which the Setup change was made. For example, if Einstein uses cloud-to-cloud services to make a change in Setup, the value of this field is Einstein. | keyword | -| salesforce.setup_audit_trail.created_by_id | Unknown. | keyword | +| salesforce.setup_audit_trail.created_by_id | The id under which the Setup change was made. For example, if Einstein uses cloud-to-cloud services to make a change in Setup, the value of this field is id of Einstein. | keyword | | salesforce.setup_audit_trail.created_by_issuer | Reserved for future use. | keyword | -| salesforce.setup_audit_trail.delegate_user | The Login-As user who executed the action in Setup. If a Login-As user didn’t perform the action, this field is blank. This field is available in API version 35.0 and later. | keyword | -| salesforce.setup_audit_trail.display | The full description of changes made in Setup. For example, if the Action field has a value of PermSetCreate, the Display field has a value like “Created permission set MAD: with user license Salesforce.” | keyword | +| salesforce.setup_audit_trail.delegate_user | The Login-As user who executed the action in Setup. If a Login-As user didn't perform the action, this field is empty string. This field is available in API version 35.0 and later. | keyword | +| salesforce.setup_audit_trail.display | The full description of changes made in Setup. For example, if the event.action field has a value of PermSetCreate, the Display field has a value like "Created permission set MAD: with user license Salesforce." | keyword | | salesforce.setup_audit_trail.event_type | Event type. | keyword | -| salesforce.setup_audit_trail.responsible_namespace_prefix | Unknown. | keyword | | salesforce.setup_audit_trail.section | The section in the Setup menu where the action occurred. For example, Manage Users or Company Profile. | keyword | | tags | List of keywords used to tag each event. | keyword | | user.id | Unique identifier of the user. | keyword | From eb9e4fad7d3473bbabde3a9988a12207a2d95bf9 Mon Sep 17 00:00:00 2001 From: kush-elastic Date: Mon, 9 Jan 2023 16:12:19 +0530 Subject: [PATCH 7/7] Address Review Comment --- packages/salesforce/_dev/build/docs/README.md | 84 +++++++++---------- packages/salesforce/docs/README.md | 84 +++++++++---------- 2 files changed, 84 insertions(+), 84 deletions(-) diff --git a/packages/salesforce/_dev/build/docs/README.md b/packages/salesforce/_dev/build/docs/README.md index 577ff16e701..6ebdba21795 100644 --- a/packages/salesforce/_dev/build/docs/README.md +++ b/packages/salesforce/_dev/build/docs/README.md @@ -2,67 +2,67 @@ ## Overview -The Salesforce integration allows you to monitor a [Salesforce](https://www.salesforce.com/) instance. Salesforce is a customer relationship management (CRM) platform. It provides an ecosystem for businesses to manage marketing, sales, commerce, service, and IT teams from anywhere with one integrated CRM platform. +The Salesforce integration allows users to monitor a [Salesforce](https://www.salesforce.com/) instance. Salesforce is a customer relationship management (CRM) platform. It provides an ecosystem for businesses to manage marketing, sales, commerce, service, and IT teams from anywhere with one integrated CRM platform. Use the Salesforce integration to: -- Gain insights into login and other operational activities by the users of your organization. +- Gain insights into login and other operational activities by the users of the organization. - Create visualizations to monitor, measure and analyze the usage trend and key data, and derive business insights. - Create alerts to reduce the MTTD and also the MTTR by referencing relevant logs when troubleshooting an issue. -As an example, you can use the data from this integration to understand the activity patterns of users based on region or the distribution of users by license type. +As an example, users can use the data from this integration to understand the activity patterns of users based on region or the distribution of users by license type. ## Data streams The Salesforce integration collects log events using the REST API of Salesforce. -**Logs** help you keep a record of events happening in Salesforce. +**Logs** help users to keep a record of events happening in Salesforce. Log data streams collected by the Salesforce integration include [Login](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_login.htm), [Logout](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_logout.htm) and [SetupAuditTrail](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_setupaudittrail.htm). Data streams: - `login_rest`: Tracks login activity of users who log in to Salesforce. - `logout_rest`: Tracks logout activity of users who logout from Salesforce. -- `setupaudittrail`: Represents changes you or other admins made in your organization's Setup area for at least the last 180 days. +- `setupaudittrail`: Represents changes users made in the user's organization's Setup area for at least the last 180 days. ## Compatibility This integration has been tested against Salesforce `Spring '22 (v54.0) release`. -In order to find out the Salesforce version of your Instance, see below: +In order to find out the Salesforce version of the user's instance, see below: -1. On the Home tab in Salesforce Classic, in the top right corner of the screen is a link to releases like `Summer '22`. This indicates your release. +1. On the Home tab in Salesforce Classic, in the top right corner of the screen is a link to releases like `Summer '22`. This indicates the release version of the salesforce instance. 2. An alternative way to find out the version of Salesforce is by hitting the following URL: - - Format: (Salesforce Instance URL)/services/data - - Example: `https://na9.salesforce.com/services/data` + - Format: (Salesforce Instance URL)/services/data + - Example: `https://na9.salesforce.com/services/data` Example response: ```xml - - - /services/data/v53.0 - 53.0 - - - - /services/data/v54.0 - 54.0 - - - - /services/data/v55.0 - 55.0 - + + + /services/data/v53.0 + 53.0 + + + + /services/data/v54.0 + 54.0 + + + + /services/data/v55.0 + 55.0 + ``` -The last one on the list is the release of your instance. In the example above, the version is `Summer '22` i.e. `v55.0`. +The last one on the list is the release of the user's salesforce instance. In the example above, the version is `Summer '22` i.e. `v55.0`. ## Prerequisites -You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. -You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended or self-manage the Elastic Stack on your own hardware. +Users need Elasticsearch for storing and searching their data and Kibana for visualizing and managing it. +Users can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on their own hardware. -In your Salesforce instance, ensure that `API Enabled permission` is selected for the user profile. Follow the below steps to enable the same: +In the user's Salesforce instance, ensure that `API Enabled permission` is selected for the user profile. Follow the below steps to enable the same: 1. Go to `Setup` > `Quick Find` > `Users`, and Click on `Users`. 2. Click on the profile link associated with the `User Account` used for data collection. @@ -74,15 +74,15 @@ For step-by-step instructions on how to set up an integration, see the [Getting ## Configuration -You need the following information from your Salesforce instance to configure this integration in Elastic: +Users need the following information from the user's Salesforce instance to configure this integration in Elastic: ### Salesforce Instance URL -The instance your Salesforce Organization uses is indicated in the URL of your browser's address bar in Salesforce Classic. The value before 'salesforce.com' is your Salesforce Instance. +The instance the user's Salesforce Organization uses is indicated in the URL of the address bar in Salesforce Classic. The value before 'salesforce.com' is the user's Salesforce Instance. Example URL: `https://na9.salesforce.com/home/home.jsp` -In the above example, the value before 'salesforce.com' is your Salesforce Instance. In this example, the Salesforce Organization is located on NA9. +In the above example, the value before 'salesforce.com' is the user's Salesforce Instance. In this example, the Salesforce Organization is located on NA9. The Salesforce Instance URL is: `https://na9.salesforce.com` @@ -90,9 +90,9 @@ In Salesforce Lightning, it is available under the user name in the “View Prof ### Client Key and Client Secret for Authentication -In order to use this integration, you need to create a new Salesforce Application using OAuth. Follow the steps below to create a connected application in Salesforce: +In order to use this integration, users need to create a new Salesforce Application using OAuth. Follow the steps below to create a connected application in Salesforce: -1. Login to [Salesforce](https://login.salesforce.com/) with the same user credentials that you want to collect data with. +1. Login to [Salesforce](https://login.salesforce.com/) with the same user credentials that the user wants to collect data with. 2. Click on Setup on the top right menu bar. On the Setup page search `App Manager` in the `Search Setup` search box at the top of the page, then select `App Manager`. 3. Click *New Connected App*. 4. Provide a name for the connected application. This will be displayed in the App Manager and on its App Launcher tile. @@ -101,9 +101,9 @@ In order to use this integration, you need to create a new Salesforce Applicatio 7. Under the API (Enable OAuth Settings) section of the page, select *Enable OAuth Settings*. 8. In the Callback URL enter the Instance URL (Please refer to `Salesforce Instance URL`) 9. Select the following OAuth scopes to apply to the connected app: - - Manage user data via APIs (api). - - Perform requests at any time (refresh_token, offline_access). - - (Optional) In case of data collection, if any permission issues arise, add the Full access (full) scope. + - Manage user data via APIs (api). + - Perform requests at any time (refresh_token, offline_access). + - (Optional) In case of data collection, if any permission issues arise, add the Full access (full) scope. 10. Select *Require Secret for the Web Server Flow* to require the app's client secret in exchange for an access token. 11. Select *Require Secret for Refresh Token Flow* to require the app's client secret in the authorization request of a refresh token and hybrid refresh token flow. 12. Click Save. It may take approximately 10 minutes for the changes to take effect. @@ -122,7 +122,7 @@ Password used for authenticating the above user. ## Additional Information -Follow the steps below, in case you need to find the API version: +Follow the steps below, in case the user needs to find the API version: 1. Go to `Setup` > `Quick Find` > `Apex Classes`. 2. Click the `New` button. @@ -131,7 +131,7 @@ Follow the steps below, in case you need to find the API version: ## Validation -After the integration is successfully configured, clicking on the Assets tab of the Salesforce Integration should display a list of available dashboards. Click on the dashboard available for your configured datastream. It should be populated with the required data. +After the integration is successfully configured, clicking on the Assets tab of the Salesforce Integration should display a list of available dashboards. Click on the dashboard available for the user's configured datastream. It should be populated with the required data. ## Troubleshooting @@ -157,14 +157,14 @@ Please refer to the Prerequisites section above for more information. If the error continues follow these steps: 1. Go to `Setup` > `Quick Find` > `Manage Connected Apps`. -2. Click on the Connected App name created by you to generate the client id and client secret (Refer to Client Key and Client Secret for Authentication) under the Master Label. +2. Click on the Connected App name created by the user to generate the client id and client secret (Refer to Client Key and Client Secret for Authentication) under the Master Label. 3. Click on Edit Policies, and select `Relax IP restrictions` from the dropdown for IP Relaxation. ## Logs reference ### Login Rest -This is the `login_rest` data stream. It represents events containing details about your organization's user login history. +This is the `login_rest` data stream. It represents events containing details about the user's organization's login history. {{event "login_rest"}} @@ -172,7 +172,7 @@ This is the `login_rest` data stream. It represents events containing details ab ### Logout Rest -This is the `logout_rest` data stream. It represents events containing details about your organization's user logout history. +This is the `logout_rest` data stream. It represents events containing details about the user's organization's logout history. {{event "logout_rest"}} @@ -180,7 +180,7 @@ This is the `logout_rest` data stream. It represents events containing details a ### SetupAuditTrail -This is the `setupaudittrail` data stream. It represents changes you or other admins made in your organization's Setup area for at least the last 180 days. +This is the `setupaudittrail` data stream. It represents changes users made in the user's organization's Setup area for at least the last 180 days. {{event "setupaudittrail"}} diff --git a/packages/salesforce/docs/README.md b/packages/salesforce/docs/README.md index 56d16831c0d..345941c0495 100644 --- a/packages/salesforce/docs/README.md +++ b/packages/salesforce/docs/README.md @@ -2,67 +2,67 @@ ## Overview -The Salesforce integration allows you to monitor a [Salesforce](https://www.salesforce.com/) instance. Salesforce is a customer relationship management (CRM) platform. It provides an ecosystem for businesses to manage marketing, sales, commerce, service, and IT teams from anywhere with one integrated CRM platform. +The Salesforce integration allows users to monitor a [Salesforce](https://www.salesforce.com/) instance. Salesforce is a customer relationship management (CRM) platform. It provides an ecosystem for businesses to manage marketing, sales, commerce, service, and IT teams from anywhere with one integrated CRM platform. Use the Salesforce integration to: -- Gain insights into login and other operational activities by the users of your organization. +- Gain insights into login and other operational activities by the users of the organization. - Create visualizations to monitor, measure and analyze the usage trend and key data, and derive business insights. - Create alerts to reduce the MTTD and also the MTTR by referencing relevant logs when troubleshooting an issue. -As an example, you can use the data from this integration to understand the activity patterns of users based on region or the distribution of users by license type. +As an example, users can use the data from this integration to understand the activity patterns of users based on region or the distribution of users by license type. ## Data streams The Salesforce integration collects log events using the REST API of Salesforce. -**Logs** help you keep a record of events happening in Salesforce. +**Logs** help users to keep a record of events happening in Salesforce. Log data streams collected by the Salesforce integration include [Login](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_login.htm), [Logout](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_logout.htm) and [SetupAuditTrail](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_setupaudittrail.htm). Data streams: - `login_rest`: Tracks login activity of users who log in to Salesforce. - `logout_rest`: Tracks logout activity of users who logout from Salesforce. -- `setupaudittrail`: Represents changes you or other admins made in your organization's Setup area for at least the last 180 days. +- `setupaudittrail`: Represents changes users made in the user's organization's Setup area for at least the last 180 days. ## Compatibility This integration has been tested against Salesforce `Spring '22 (v54.0) release`. -In order to find out the Salesforce version of your Instance, see below: +In order to find out the Salesforce version of the user's instance, see below: -1. On the Home tab in Salesforce Classic, in the top right corner of the screen is a link to releases like `Summer '22`. This indicates your release. +1. On the Home tab in Salesforce Classic, in the top right corner of the screen is a link to releases like `Summer '22`. This indicates the release version of the salesforce instance. 2. An alternative way to find out the version of Salesforce is by hitting the following URL: - - Format: (Salesforce Instance URL)/services/data - - Example: `https://na9.salesforce.com/services/data` + - Format: (Salesforce Instance URL)/services/data + - Example: `https://na9.salesforce.com/services/data` Example response: ```xml - - - /services/data/v53.0 - 53.0 - - - - /services/data/v54.0 - 54.0 - - - - /services/data/v55.0 - 55.0 - + + + /services/data/v53.0 + 53.0 + + + + /services/data/v54.0 + 54.0 + + + + /services/data/v55.0 + 55.0 + ``` -The last one on the list is the release of your instance. In the example above, the version is `Summer '22` i.e. `v55.0`. +The last one on the list is the release of the user's salesforce instance. In the example above, the version is `Summer '22` i.e. `v55.0`. ## Prerequisites -You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. -You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended or self-manage the Elastic Stack on your own hardware. +Users need Elasticsearch for storing and searching their data and Kibana for visualizing and managing it. +Users can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on their own hardware. -In your Salesforce instance, ensure that `API Enabled permission` is selected for the user profile. Follow the below steps to enable the same: +In the user's Salesforce instance, ensure that `API Enabled permission` is selected for the user profile. Follow the below steps to enable the same: 1. Go to `Setup` > `Quick Find` > `Users`, and Click on `Users`. 2. Click on the profile link associated with the `User Account` used for data collection. @@ -74,15 +74,15 @@ For step-by-step instructions on how to set up an integration, see the [Getting ## Configuration -You need the following information from your Salesforce instance to configure this integration in Elastic: +Users need the following information from the user's Salesforce instance to configure this integration in Elastic: ### Salesforce Instance URL -The instance your Salesforce Organization uses is indicated in the URL of your browser's address bar in Salesforce Classic. The value before 'salesforce.com' is your Salesforce Instance. +The instance the user's Salesforce Organization uses is indicated in the URL of the address bar in Salesforce Classic. The value before 'salesforce.com' is the user's Salesforce Instance. Example URL: `https://na9.salesforce.com/home/home.jsp` -In the above example, the value before 'salesforce.com' is your Salesforce Instance. In this example, the Salesforce Organization is located on NA9. +In the above example, the value before 'salesforce.com' is the user's Salesforce Instance. In this example, the Salesforce Organization is located on NA9. The Salesforce Instance URL is: `https://na9.salesforce.com` @@ -90,9 +90,9 @@ In Salesforce Lightning, it is available under the user name in the “View Prof ### Client Key and Client Secret for Authentication -In order to use this integration, you need to create a new Salesforce Application using OAuth. Follow the steps below to create a connected application in Salesforce: +In order to use this integration, users need to create a new Salesforce Application using OAuth. Follow the steps below to create a connected application in Salesforce: -1. Login to [Salesforce](https://login.salesforce.com/) with the same user credentials that you want to collect data with. +1. Login to [Salesforce](https://login.salesforce.com/) with the same user credentials that the user wants to collect data with. 2. Click on Setup on the top right menu bar. On the Setup page search `App Manager` in the `Search Setup` search box at the top of the page, then select `App Manager`. 3. Click *New Connected App*. 4. Provide a name for the connected application. This will be displayed in the App Manager and on its App Launcher tile. @@ -101,9 +101,9 @@ In order to use this integration, you need to create a new Salesforce Applicatio 7. Under the API (Enable OAuth Settings) section of the page, select *Enable OAuth Settings*. 8. In the Callback URL enter the Instance URL (Please refer to `Salesforce Instance URL`) 9. Select the following OAuth scopes to apply to the connected app: - - Manage user data via APIs (api). - - Perform requests at any time (refresh_token, offline_access). - - (Optional) In case of data collection, if any permission issues arise, add the Full access (full) scope. + - Manage user data via APIs (api). + - Perform requests at any time (refresh_token, offline_access). + - (Optional) In case of data collection, if any permission issues arise, add the Full access (full) scope. 10. Select *Require Secret for the Web Server Flow* to require the app's client secret in exchange for an access token. 11. Select *Require Secret for Refresh Token Flow* to require the app's client secret in the authorization request of a refresh token and hybrid refresh token flow. 12. Click Save. It may take approximately 10 minutes for the changes to take effect. @@ -122,7 +122,7 @@ Password used for authenticating the above user. ## Additional Information -Follow the steps below, in case you need to find the API version: +Follow the steps below, in case the user needs to find the API version: 1. Go to `Setup` > `Quick Find` > `Apex Classes`. 2. Click the `New` button. @@ -131,7 +131,7 @@ Follow the steps below, in case you need to find the API version: ## Validation -After the integration is successfully configured, clicking on the Assets tab of the Salesforce Integration should display a list of available dashboards. Click on the dashboard available for your configured datastream. It should be populated with the required data. +After the integration is successfully configured, clicking on the Assets tab of the Salesforce Integration should display a list of available dashboards. Click on the dashboard available for the user's configured datastream. It should be populated with the required data. ## Troubleshooting @@ -157,14 +157,14 @@ Please refer to the Prerequisites section above for more information. If the error continues follow these steps: 1. Go to `Setup` > `Quick Find` > `Manage Connected Apps`. -2. Click on the Connected App name created by you to generate the client id and client secret (Refer to Client Key and Client Secret for Authentication) under the Master Label. +2. Click on the Connected App name created by the user to generate the client id and client secret (Refer to Client Key and Client Secret for Authentication) under the Master Label. 3. Click on Edit Policies, and select `Relax IP restrictions` from the dropdown for IP Relaxation. ## Logs reference ### Login Rest -This is the `login_rest` data stream. It represents events containing details about your organization's user login history. +This is the `login_rest` data stream. It represents events containing details about the user's organization's login history. An example event for `login_rest` looks as following: @@ -335,7 +335,7 @@ An example event for `login_rest` looks as following: ### Logout Rest -This is the `logout_rest` data stream. It represents events containing details about your organization's user logout history. +This is the `logout_rest` data stream. It represents events containing details about the user's organization's logout history. An example event for `logout_rest` looks as following: @@ -491,7 +491,7 @@ An example event for `logout_rest` looks as following: ### SetupAuditTrail -This is the `setupaudittrail` data stream. It represents changes you or other admins made in your organization's Setup area for at least the last 180 days. +This is the `setupaudittrail` data stream. It represents changes users made in the user's organization's Setup area for at least the last 180 days. An example event for `setupaudittrail` looks as following: