Skip to content

Fix ironbank validation in 8.8#36115

Closed
reakaleek wants to merge 1 commit intoelastic:8.8from
reakaleek:feature/fix-ironbank-validation
Closed

Fix ironbank validation in 8.8#36115
reakaleek wants to merge 1 commit intoelastic:8.8from
reakaleek:feature/fix-ironbank-validation

Conversation

@reakaleek
Copy link
Member

@reakaleek reakaleek commented Jul 19, 2023 •
edited
Loading

What does this PR do?

Downgrades PyYaml to 5.3.1 in libbeat/tests/system/requirements.txt to fix ironbank validation in 8.8.

Successul Jenkins build: https://beats-ci.elastic.co/job/Beats/job/ironbank-validation-mbp/view/change-requests/job/PR-36115/1/

Why is it important?

The jenkins job https://beats-ci.elastic.co/job/Beats/job/ironbank-validation-mbp/job/8.8/57/ is failing with the error:

      AttributeError: cython_sources
full error message 
/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!
      
              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.
      
              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.
      
              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************
      
      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/Users/jcal/Projects/beats/libbeat/tests/system/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/jcal/Projects/beats/libbeat/tests/system/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/jcal/Projects/beats/libbeat/tests/system/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 314, in run
          self.find_sources()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
          mm.run()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 551, in run
          self.add_defaults()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
          sdist.add_defaults(self)
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
          super().add_defaults()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

cd libbeat/tests/system
virtualenv .venv
source .venv/bin/activate
pip install -r requirements.txt

Related issues

Use cases

Screenshots

Logs

/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!
      
              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.
      
              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.
      
              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************
      
      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/Users/jcal/Projects/beats/libbeat/tests/system/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/jcal/Projects/beats/libbeat/tests/system/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/jcal/Projects/beats/libbeat/tests/system/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 314, in run
          self.find_sources()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
          mm.run()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 551, in run
          self.add_defaults()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 589, in add_defaults
          sdist.add_defaults(self)
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/command/sdist.py", line 104, in add_defaults
          super().add_defaults()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/private/var/folders/y_/xh6pwfy15ns98xs2krq551gm0000gn/T/pip-build-env-5ecqkc4d/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources

@reakaleek
Downgrade PyYaml to 5.3.1
80ce812 
Upgrading to 6.0.1 did not work because there is a dependency conflict with docker-compose.

When using the version 5.4.1 the following error message appears when trying to run `pip install -r requirements.txt`

```
AttributeError: cython_source
```

Related issue: yaml/pyyaml#601
@reakaleek reakaleek requested review from a team as code owners July 19, 2023 15:07
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 19, 2023
@reakaleek reakaleek requested a review from leehinman July 19, 2023 15:07
@botelastic botelastic bot added Team:Automation Label for the Observability productivity team and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 19, 2023
@reakaleek reakaleek changed the base branch from main to 8.8 July 19, 2023 15:07
@reakaleek reakaleek requested a review from a team July 19, 2023 15:15
@elasticmachine
Copy link
Contributor

elasticmachine commented Jul 19, 2023 •
edited by jenkins-beats-ci bot
Loading

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-07-19T15:08:10.247+0000

  • Duration: 82 min 45 sec

Test stats 🧪

Test Results
Failed 0
Passed 25469
Skipped 1796
Total 27265

Steps errors 13

Expand to view the steps failures

Show only the first 10 steps failures

metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 2 min 47 sec . View more details here
  • Description: mage pythonIntegTest
metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 0 min 27 sec . View more details here
  • Description: mage pythonIntegTest
metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 0 min 27 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/metricbeat-goIntegTest - mage goIntegTest
  • Took 5 min 48 sec . View more details here
  • Description: mage goIntegTest
x-pack/metricbeat-goIntegTest - mage goIntegTest
  • Took 4 min 22 sec . View more details here
  • Description: mage goIntegTest
x-pack/metricbeat-goIntegTest - mage goIntegTest
  • Took 4 min 59 sec . View more details here
  • Description: mage goIntegTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 1 min 6 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 0 min 21 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 0 min 21 sec . View more details here
  • Description: mage pythonIntegTest
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@vit-zikmund
Copy link

vit-zikmund commented Jul 19, 2023

Hey folks, just cruising by from yaml/pyyaml#601. PyYAML 5.3.1 has a security hole, you should be better off with the latest and at least still maintained version 6.0.1.

@kuisathaverat kuisathaverat self-requested a review July 19, 2023 18:17
kuisathaverat
kuisathaverat requested changes Jul 19, 2023
View reviewed changes
Copy link
Contributor

@kuisathaverat kuisathaverat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested it locally it should work with PyYAML=5.4.1

@reakaleek reakaleek closed this Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kuisathaverat kuisathaverat kuisathaverat requested changes

@gsantoro gsantoro Awaiting requested review from gsantoro gsantoro was automatically assigned from elastic/obs-cloudnative-monitoring

@constanca-m constanca-m Awaiting requested review from constanca-m constanca-m was automatically assigned from elastic/obs-cloudnative-monitoring

@ycombinator ycombinator Awaiting requested review from ycombinator ycombinator is a code owner automatically assigned from elastic/elastic-agent-data-plane

@leehinman leehinman Awaiting requested review from leehinman leehinman is a code owner automatically assigned from elastic/elastic-agent-data-plane

Assignees

@reakaleek reakaleek

Labels

Team:Automation Label for the Observability productivity team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@reakaleek @elasticmachine @vit-zikmund @kuisathaverat