Correctly inject dependency list when packaging using DROP_PATH#7795
Merged
pchila merged 1 commit intoelastic:mainfrom Apr 9, 2025
Merged
Correctly inject dependency list when packaging using DROP_PATH#7795pchila merged 1 commit intoelastic:mainfrom
pchila merged 1 commit intoelastic:mainfrom
Conversation
pchila
added
bug
Contributor
|
Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane) |
Contributor
|
Pinging @elastic/elastic-agent (Team:Elastic-Agent) |
Contributor
|
This pull request does not have a backport label. Could you fix it @pchila? 🙏
|
Member
Author
|
elastic-agent-package pipeline dry-run: https://buildkite.com/elastic/elastic-agent-package/builds/5251 |
Contributor
pkoutsovasilis
left a comment
pkoutsovasilis
left a comment
There was a problem hiding this comment.
Ok I see what was the issue before and why we hit that only in the DRA manifest building. But I believe that this is addressed now (looking here), waiting for the CI to be green 🙂
|
Contributor
💚 Build Succeeded
cc @pchila |
pkoutsovasilis
approved these changes
Apr 9, 2025
pchila
added a commit
that referenced
this pull request
Apr 10, 2025
3 tasks
pchila
added a commit
that referenced
this pull request
Apr 10, 2025
pchila
added a commit
that referenced
this pull request
Apr 10, 2025
pchila
added a commit
that referenced
this pull request
Apr 11, 2025
pchila
added a commit
that referenced
this pull request
Apr 16, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
pkoutsovasilis
pushed a commit
that referenced
this pull request
Aug 7, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> # Conflicts: # .buildkite/integration.pipeline.yml # dev-tools/mage/checksums.go # dev-tools/mage/dockerbuilder.go # dev-tools/mage/manifest/manifest.go # dev-tools/mage/pkgtypes.go # dev-tools/mage/settings.go # dev-tools/packaging/packages.yml # dev-tools/packaging/testing/package_test.go # testing/integration/ess/upgrade_standalone_same_commit_test.go
pkoutsovasilis
pushed a commit
that referenced
this pull request
Aug 7, 2025
pkoutsovasilis
pushed a commit
that referenced
this pull request
Aug 7, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
pkoutsovasilis
pushed a commit
that referenced
this pull request
Aug 7, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
pkoutsovasilis
pushed a commit
that referenced
this pull request
Aug 7, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
pkoutsovasilis
pushed a commit
that referenced
this pull request
Aug 7, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
pkoutsovasilis
pushed a commit
that referenced
this pull request
Aug 7, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
pkoutsovasilis
added a commit
that referenced
this pull request
Aug 8, 2025
* Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * ci: build agent from snapshot DRA (#9048) * feat: rework .package-version and mage integration:UpdatePackageVersion to make CI build always from snapshot DRA * feat: incorporate USE_PACKAGE_VERSION in mage * experiment: bump version.go * Revert "experiment: bump version.go" This reverts commit a57ee10. * chore: bump .package-version * feat: allow AGENT_VERSION to be overridden by env var * fix: use named args for all args in integration_tests_tf.ps1 * feat: panic on err of initPackageVersion * fix: don't panic when .package-version file doesn't exist, log it instead * feat: rework fabrication of CI_ELASTIC_AGENT_DOCKER_IMAGE * feat: use os.WriteFile in writePackageVersion * chore: bump to latest snapshot DRA * fix: always DownloadManifest if PackagingFromManifest is set in mage package * fix: check err of filepath.Abs(dropPath) * fix: remove fips-related changes * fix: resolve QF1004: could use strings.ReplaceAll linter errors * fix: remove unused imports --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co>
pkoutsovasilis
added a commit
that referenced
this pull request
Aug 8, 2025
* ci: build agent from snapshot DRA (#9048) * feat: rework .package-version and mage integration:UpdatePackageVersion to make CI build always from snapshot DRA * feat: incorporate USE_PACKAGE_VERSION in mage * experiment: bump version.go * Revert "experiment: bump version.go" This reverts commit a57ee10. * chore: bump .package-version * feat: allow AGENT_VERSION to be overridden by env var * fix: use named args for all args in integration_tests_tf.ps1 * feat: panic on err of initPackageVersion * fix: don't panic when .package-version file doesn't exist, log it instead * feat: rework fabrication of CI_ELASTIC_AGENT_DOCKER_IMAGE * feat: use os.WriteFile in writePackageVersion * chore: bump to latest snapshot DRA * fix: always DownloadManifest if PackagingFromManifest is set in mage package * fix: check err of filepath.Abs(dropPath) (cherry picked from commit a155660) # Conflicts: # .buildkite/integration.pipeline.yml # .buildkite/scripts/steps/ess.ps1 # .package-version # dev-tools/mage/manifest/manifest.go * [8.x](backport #7690) Fips packaging (#7790) * Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * fix: 8.18.5 snapshot DRA * fix: resolve QF1004: could use strings.ReplaceAll linter errors --------- Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
pkoutsovasilis
added a commit
that referenced
this pull request
Aug 8, 2025
* ci: build agent from snapshot DRA (#9048) * feat: rework .package-version and mage integration:UpdatePackageVersion to make CI build always from snapshot DRA * feat: incorporate USE_PACKAGE_VERSION in mage * experiment: bump version.go * Revert "experiment: bump version.go" This reverts commit a57ee10. * chore: bump .package-version * feat: allow AGENT_VERSION to be overridden by env var * fix: use named args for all args in integration_tests_tf.ps1 * feat: panic on err of initPackageVersion * fix: don't panic when .package-version file doesn't exist, log it instead * feat: rework fabrication of CI_ELASTIC_AGENT_DOCKER_IMAGE * feat: use os.WriteFile in writePackageVersion * chore: bump to latest snapshot DRA * fix: always DownloadManifest if PackagingFromManifest is set in mage package * fix: check err of filepath.Abs(dropPath) (cherry picked from commit a155660) # Conflicts: # .buildkite/integration.pipeline.yml # .buildkite/scripts/steps/ess.ps1 # .package-version # dev-tools/mage/manifest/manifest.go * [8.x](backport #7690) Fips packaging (#7790) * Make components in packages configurable (#7602) * Redefine ExpectedBinaries as YAML config * Move ExpectedBinaries closer to package spec file * Fix error formatting in downloadDRAArtifacts * add packageName template to ExpectedPackages * use a relaxed dependencies version for IAR releases * Remove FIPS hack introduced in PR #7486 * Allow for a looser match on relaxing dependencies versions * Add debug logging when packaging with EXTERNAL=true * move package tests to dedicated package * Fips packaging (#7690) * Add component list to specs * extract component dependencies from the packages to be built * Refactor component extraction from package specs * Fix package tests error handling * Inject dependencies and remove references to ExpectedBinaries * Remove ExpectedBinaries global * Add rootdir to components * Extract actual version matched on the package file and use it to render RootDir * Package elastic-agent FIPS specs when FIPS=true is specified * refactor ResolveManifestPackage * Move FIPS compile settings in packages.yml * Add more FIPS components * Properly handle dependenciesVersion when calling mage package * Refactor ChecksumsWithoutManifest to use list of dependencies instead of globbing files * Rework useDRAAgentBinaryForPackage for repackaging agent Define elastic-agent-core components (both FIPS and non-FIPS variants) and define package name and root dir templates. Implement some filtering on component list to extract the correct component definition according to the FIPSBuild flag. Refactor code that downloads pre-compiled elastic-agent binaries and places them in the golangCrossBuild directory to make use of the new component definition. * Write spec FIPS flag into manifest.yaml when packaging * Add FIPS elastic agent basic and cloud docker images * Build FIPS docker images in CI packaging * Fix FIPS .tar.gz package tests * Restructure package tests * Extend FIPS check to all binaries in components directory * Create FIPS elastic-agent-core artifacts in elastic-agent-binary-dra pipeline * Cleanup ChecksumsWithManifest and improve godoc * Improve godoc for BinarySpec * Correctly inject dependency list when packaging using DROP_PATH (#7795) * Restore qualifier=core for elastic-agent-core packaging specs (#7805) Restore qualifier for elastic-agent-core packaging specs to avoid changing the rootDir name of the archives. The qualifier had been removed in PR #7690 trying to use the spec name: this worked to get the desired file name but changed the root Dir name which uses '{{.BeatName}}{{if .Qualifier}}-{{.Qualifier}}{{end}}' in the template definition instead of '{{.Name}}' which would render the spec name. * Modify fips core spec qualifier and name (#7818) * Reintroduce cloud-defend component * Filter components by package-type --------- Co-authored-by: Paolo Chilà <paolo.chila@elastic.co> * fix: point .package-version to latest 8.17 SNAPSHOT DRA * fix: resolve QF1004: could use strings.ReplaceAll linter errors * fix: remove unused var hintsInputsDFilePattern --------- Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Paolo Chilà <paolo.chila@elastic.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fix a bug introduced by #7690 where dependency list is not correctly passed to
isPlatformIndependentPackage().Without dependency list
isPlatformIndependentPackage()did not recognizeconnectors-9.1.0-SNAPSHOT.zipas a platform independent package so it wasn't copied in thearchivesdirectory.This in turn would make the
elastic-agent-servicedocker image spec fail because of these lines that expects the connectors zip to exist in the archive directory.Why is it important?
To fix unified release for elastic-agent
Checklist
[ ] I have commented my code, particularly in hard-to-understand areas[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration files[ ] I have added tests that prove my fix is effective or that my feature works[ ] I have added an entry in./changelog/fragmentsusing the changelog tool[ ] I have added an integration test or an E2E testDisruptive User Impact
None
How to test this PR locally
Related issues
Questions to ask yourself