Skip to content

Conversation

@adriansr
Copy link
Contributor

@adriansr adriansr commented Jan 10, 2022

What does this PR do?

Restores validation for @timestamp field in Filebeat's modules tests. This includes re-generating the expected files as they have been out of date for a while, and some of them had their @timestamp field lost since they've been updated.

Why is it important?

Since #28450, @timestamp validation was disabled for Filebeat modules. Restoring it ensures we catch bugs related to extracting the correct timestamp from logs.

Note: This temporarily disables validation for ibmmq.errorlog, as there's a bug parsing timestamps in that fileset. This will be fixed in a separate PR. (here #29773)

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

This restores the `@timestamp` field in Filebeat's module tests, so that
it is properly validated.
This fileset is failing to extract the correct timestamp from the logs.
This will be fixed in a separate PR.
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 10, 2022
@mergify
Copy link
Contributor

mergify bot commented Jan 10, 2022

This pull request does not have a backport label. Could you fix it @adriansr? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Jan 10, 2022
@adriansr adriansr changed the title Filebeat modules: Restore @timestamp field validation Filebeat tests: Restore @timestamp field validation Jan 10, 2022
@adriansr adriansr mentioned this pull request Jan 10, 2022
6 tasks
@elasticmachine
Copy link
Contributor

elasticmachine commented Jan 10, 2022

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-01-10T13:44:39.235+0000

  • Duration: 89 min 37 sec

  • Commit: 4ee80ce

Test stats 🧪

Test Results
Failed 0
Passed 10277
Skipped 1104
Total 11381

Steps errors 8

Expand to view the steps failures

filebeat-windows-2016-windows-2016 - mage build unitTest
  • Took 2 min 46 sec . View more details here
  • Description: mage build unitTest
filebeat-windows-2016-windows-2016 - mage build unitTest
  • Took 1 min 34 sec . View more details here
  • Description: mage build unitTest
filebeat-windows-2016-windows-2016 - mage build unitTest
  • Took 1 min 33 sec . View more details here
  • Description: mage build unitTest
gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beat
  • Took 0 min 3 sec . View more details here
  • Description: @echo off gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beats-ci-temp/Beats/beats/PR-29772-2
x-pack/filebeat-windows-10-windows-10 - mage build unitTest
  • Took 5 min 28 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-windows-10-windows-10 - mage build unitTest
  • Took 3 min 22 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-windows-10-windows-10 - mage build unitTest
  • Took 3 min 20 sec . View more details here
  • Description: mage build unitTest
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@adriansr adriansr added backport-v8.0.0 Automated backport with mergify Team:Integrations Label for the Integrations team and removed backport-skip Skip notification from the automated backport with mergify labels Jan 10, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 10, 2022
@adriansr
Copy link
Contributor Author

CI failing under Windows recently due to python venv issues. Merging anyway.

@adriansr adriansr merged commit bcb83e8 into elastic:master Jan 10, 2022
mergify bot pushed a commit that referenced this pull request Jan 10, 2022
This restores the `@timestamp` field in Filebeat's module tests, so that
it is properly validated.

Temporarily disable timestamp validation in ibmmq module

(cherry picked from commit bcb83e8)
adriansr added a commit that referenced this pull request Jan 10, 2022
This restores the `@timestamp` field in Filebeat's module tests, so that
it is properly validated.

Temporarily disable timestamp validation in ibmmq module

(cherry picked from commit bcb83e8)

Co-authored-by: Adrian Serrano <[email protected]>
v1v added a commit to v1v/beats that referenced this pull request Jan 12, 2022
…b-for-macos

* upstream/master: (172 commits)
  [Elastic Agent] Fix issue with ensureServiceToken. (elastic#29800)
  [Winlogbeat] Add provider name to Security routing pipeline check (elastic#29781)
  Add summary to journeys which don't emit journey:end (early node subprocess exits) (elastic#29606)
  Prepare 8.0.0-rc1 changelog (elastic#29795) (elastic#29806)
  Change docker image from CentOS 7 to Ubuntu 20.04 (elastic#29681)
  libbeat/processors/add_process_metadata: implement a process cache eviction policy (elastic#29717)
  [Automation] Update elastic stack version to 8.1.0-7004acda for testing (elastic#29783)
  Missing changelog entry for elastic#29773 (elastic#29791)
  Add a readme for k8s autodiscover provider (elastic#28213)
  Remove overriding of index pattern on the Kubernetes overview dashboard (elastic#29676)
  jjbb: remove obsoleted branches (<7.16) (elastic#29707)
  Add k8s metadata in state_cronjob metricset (elastic#29572)
  ibmmq: Fix timestamp parsing (elastic#29773)
  Do not add date to index if `@meta.index` is set (elastic#29775)
  ci: uses aliases for the branches (elastic#29706)
  Filebeat tests: Restore `@timestamp` field validation (elastic#29772)
  Forward port 7.16.3 changelog to master (elastic#29777)
  auditd: Store program arguments in process.args array (elastic#29601)
  System/socket: Support kernel_clone() replacement for _do_fork() (elastic#29744)
  Do not mention removal if version is not specified in `cfgwarn` messages (elastic#29727)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-v8.0.0 Automated backport with mergify Filebeat Filebeat review Team:Integrations Label for the Integrations team :Testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants