Skip to content

Generate an integrations package from a release#983

Merged
brokensound77 merged 14 commits intoelastic:mainfrom
brokensound77:generate-oob-package
Mar 9, 2021
Merged

Generate an integrations package from a release#983
brokensound77 merged 14 commits intoelastic:mainfrom
brokensound77:generate-oob-package

Conversation

@brokensound77
Copy link
Copy Markdown
Contributor

@brokensound77 brokensound77 commented Mar 5, 2021

Issues

related to elastic/package-storage#843

Summary

This generates a package in the format expected by package-storage for use with the integrations process. The package is generated along with the other artifacts from Packaging and can be built with make release or dev build-release.

To see this with WIP changelog code along with more comprehensive changes, refer to this diff.

To see the package built with this PR, refer to the checks

@brokensound77 brokensound77 added the python Internal python for the repository label Mar 5, 2021
Comment thread detection_rules/misc.py
Comment thread detection_rules/schemas/definitions.py
Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/utils.py Outdated
Comment thread etc/rules-changelog.json Outdated
Comment thread requirements.txt
Comment thread tests/test_packages.py Outdated
Comment thread detection_rules/packaging.py Outdated
Comment thread detection_rules/packaging.py Outdated
UUID_PATTERN = r'[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}'

_version = r'\d+\.\d+(\.\d+[\w-]*)*'
CONDITION_VERSION_PATTERN = rf'^\^{_version}$'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we'll loosen this up as we go. might move from ^ to ~ or something.
no changes needed here, just an fyi

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to know - can always expand it here

Comment thread detection_rules/schemas/definitions.py
Comment thread detection_rules/schemas/definitions.py Outdated
Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/semver.py Outdated
update_version_lock=False, registry_data: dict = None, verbose=True):
"""Initialize a package."""
self.rules: List[Rule] = [r.copy() for r in rules]
self.rules = [r.copy() for r in rules]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like type inspection has figured it out. I think we could annotate the Rule.copy method to explicitly mention a return type of Rule, but PyCharm at least is pretty smart

Comment thread detection_rules/packaging.py Outdated
Comment thread detection_rules/packaging.py Outdated
Comment thread detection_rules/packaging.py Outdated

assert self.registry_data

manifest = RegistryPackageManifest.from_dict(self.registry_data)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would make more sense to give this class an already assembled RegistryPackageManifest, but we can do this later with a restructur

Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/semver.py Outdated
Comment thread detection_rules/schemas/registry_package.py Outdated
Comment thread detection_rules/schemas/registry_package.py
@threat-punter threat-punter force-pushed the generate-oob-package branch from 10c6cbe to 4117d74 Compare March 9, 2021 18:38
Comment thread detection_rules/packaging.py
Copy link
Copy Markdown
Contributor

@threat-punter threat-punter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one, @brokensound77. I left one comment for your review. LGTM!

Comment thread detection_rules/schemas/definitions.py Outdated
Comment thread detection_rules/packaging.py Outdated
Comment thread detection_rules/packaging.py Outdated
Copy link
Copy Markdown
Contributor

@rw-access rw-access left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove "and dependencies (e.g. ML jobs)" from the readme_text and this LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Internal python for the repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants