Skip to content

Cisco Umbrella: Fix umbrella dns logs populating destination.ip instead of source.nat.ip#31454

Merged
adriansr merged 1 commit intoelastic:mainfrom
adriansr:fix_umbrella_dns_nat
Apr 28, 2022
Merged

Cisco Umbrella: Fix umbrella dns logs populating destination.ip instead of source.nat.ip#31454
adriansr merged 1 commit intoelastic:mainfrom
adriansr:fix_umbrella_dns_nat

Conversation

@adriansr
Copy link
Contributor

@adriansr adriansr commented Apr 28, 2022 •
edited
Loading

What does this PR do?

Changes the Cisco Umbrella ingest pipeline to populate the source.nat.ip field instead of destination.ip.

Also fixes a format error in the first two sample logs.

Why is it important?

The dataset was using the wrong field, as the documented meaning of the source column is "External IP", not a destination IP.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Notes to reviewers

  • There was a format error in the sample DNS logs. The identities column, a string containing comma-separated values, was mistakenly split into multiple columns, causing parsing errors. That's why this PR seems to modify more values than it should.
  • I've taken the opportunity to replace the use of some grok processors to parse IP addresses with convert(type: ip), as this is going to be backported only into versions where type: ip is supported.

@adriansr
Umbrella: Correctly populate nat address field
4028de6 
This patch corrects the Cisco Umbrella DNS logs pipeline to populate the
source.nat.ip field instead of the destination.address field.

Also fixes a mistake on the testing logs.
@adriansr adriansr requested a review from a team as a code owner April 28, 2022 13:57
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 28, 2022
@elasticmachine
Copy link
Contributor

elasticmachine commented Apr 28, 2022

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 28, 2022
@adriansr adriansr added bug needs_team Indicates that the issue/PR needs a Team:* label labels Apr 28, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 28, 2022
@adriansr adriansr added needs_team Indicates that the issue/PR needs a Team:* label needs_integration_sync Changes in this PR need synced to elastic/integrations. backport-v8.2.0 Automated backport with mergify backport-7.17 Automated backport to the 7.17 branch with mergify labels Apr 28, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Apr 28, 2022
@botelastic
Copy link

botelastic bot commented Apr 28, 2022

This pull request doesn't have a Team:<team> label.

@adriansr adriansr changed the title Cisco Umbrella: Correctly populate nat address field Cisco Umbrella: Populate client.nat.ip field instead of destination.ip Apr 28, 2022
@adriansr adriansr changed the title Cisco Umbrella: Populate client.nat.ip field instead of destination.ip Cisco Umbrella: Fix umbrella dns logs populating destination.ip instead of source.nat.ip Apr 28, 2022
@elasticmachine
Copy link
Contributor

elasticmachine commented Apr 28, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-04-28T13:57:33.894+0000

  • Duration: 69 min 12 sec

Test stats 🧪

Test Results
Failed 0
Passed 2070
Skipped 159
Total 2229

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@adriansr adriansr mentioned this pull request Apr 28, 2022
Merged
4 tasks
@adriansr adriansr merged commit 9eca3a2 into elastic:main Apr 28, 2022
@adriansr adriansr deleted the fix_umbrella_dns_nat branch April 28, 2022 16:03
@mergify mergify bot mentioned this pull request Apr 28, 2022
Merged
mergify bot pushed a commit that referenced this pull request Apr 28, 2022
@adriansr @mergify
Umbrella: Correctly populate nat address field (#31454)
12f87fa 
This patch corrects the Cisco Umbrella DNS logs pipeline to populate the
source.nat.ip field instead of the destination.address field.

Also fixes a mistake on the testing logs.

(cherry picked from commit 9eca3a2)
@mergify mergify bot mentioned this pull request Apr 28, 2022
Merged
adriansr added a commit that referenced this pull request Apr 29, 2022
@adriansr
Umbrella: Correctly populate nat address field (#31454)
3f6b4ad 
This patch corrects the Cisco Umbrella DNS logs pipeline to populate the
source.nat.ip field instead of the destination.address field.

Also fixes a mistake on the testing logs.

(cherry picked from commit 9eca3a2)
adriansr added a commit that referenced this pull request Apr 29, 2022
@mergify @adriansr
[8.2](backport #31454) Cisco Umbrella: Fix umbrella dns logs populati…
d1e4422 
…ng destination.ip instead of source.nat.ip (#31459)

This patch corrects the Cisco Umbrella DNS logs pipeline to populate the
source.nat.ip field instead of the destination.address field.

Also fixes a mistake on the testing logs.

(cherry picked from commit 9eca3a2)

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
adriansr added a commit that referenced this pull request Apr 29, 2022
@mergify @adriansr
Umbrella: Correctly populate nat address field (#31454) (#31458)
1a30a2f 
This patch corrects the Cisco Umbrella DNS logs pipeline to populate the
source.nat.ip field instead of the destination.address field.

Also fixes a mistake on the testing logs.

(cherry picked from commit 9eca3a2)

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
kush-elastic pushed a commit to kush-elastic/beats that referenced this pull request May 2, 2022
@adriansr @kush-elastic
Umbrella: Correctly populate nat address field (elastic#31454)
23f2b53 
This patch corrects the Cisco Umbrella DNS logs pipeline to populate the
source.nat.ip field instead of the destination.address field.

Also fixes a mistake on the testing logs.
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
@adriansr @chrisberkhout
Umbrella: Correctly populate nat address field (#31454)
a590f71 
This patch corrects the Cisco Umbrella DNS logs pipeline to populate the
source.nat.ip field instead of the destination.address field.

Also fixes a mistake on the testing logs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@P1llus P1llus P1llus approved these changes

Assignees

@adriansr adriansr

Labels

backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.2.0 Automated backport with mergify bug needs_integration_sync Changes in this PR need synced to elastic/integrations. review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@adriansr @elasticmachine @P1llus