From 889a75902efd0abd5550c36c176622b553d1f18a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20=C5=9Awi=C4=85tek?= Date: Fri, 28 Mar 2025 13:43:29 +0100 Subject: [PATCH 1/3] Support replace directives in otel readme mage command (#7456) * Support pseudo-versions in otel readme mage command * Move otel dependency handling to a separate package (cherry picked from commit 90712540287953f7c2a880965ae61951b1469a7e) --- NOTICE.txt | 82 +++++++------- dev-tools/mage/otel/deps.go | 186 +++++++++++++++++++++++++++++++ dev-tools/mage/otel/deps_test.go | 145 ++++++++++++++++++++++++ go.mod | 6 +- go.sum | 8 +- magefile.go | 150 +------------------------ 6 files changed, 382 insertions(+), 195 deletions(-) create mode 100644 dev-tools/mage/otel/deps.go create mode 100644 dev-tools/mage/otel/deps_test.go diff --git a/NOTICE.txt b/NOTICE.txt index 3dca80cb261..0c298dce444 100644 --- a/NOTICE.txt +++ b/NOTICE.txt @@ -17973,11 +17973,11 @@ THE SOFTWARE. -------------------------------------------------------------------------------- Dependency : golang.org/x/crypto -Version: v0.36.0 +Version: v0.33.0 Licence type (autodetected): BSD-3-Clause -------------------------------------------------------------------------------- -Contents of probable licence file $GOMODCACHE/golang.org/x/crypto@v0.36.0/LICENSE: +Contents of probable licence file $GOMODCACHE/golang.org/x/crypto@v0.33.0/LICENSE: Copyright 2009 The Go Authors. @@ -18045,13 +18045,50 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +-------------------------------------------------------------------------------- +Dependency : golang.org/x/mod +Version: v0.23.0 +Licence type (autodetected): BSD-3-Clause +-------------------------------------------------------------------------------- + +Contents of probable licence file $GOMODCACHE/golang.org/x/mod@v0.23.0/LICENSE: + +Copyright 2009 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google LLC nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + -------------------------------------------------------------------------------- Dependency : golang.org/x/net -Version: v0.38.0 +Version: v0.35.0 Licence type (autodetected): BSD-3-Clause -------------------------------------------------------------------------------- -Contents of probable licence file $GOMODCACHE/golang.org/x/net@v0.38.0/LICENSE: +Contents of probable licence file $GOMODCACHE/golang.org/x/net@v0.35.0/LICENSE: Copyright 2009 The Go Authors. @@ -106097,43 +106134,6 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. --------------------------------------------------------------------------------- -Dependency : golang.org/x/mod -Version: v0.23.0 -Licence type (autodetected): BSD-3-Clause --------------------------------------------------------------------------------- - -Contents of probable licence file $GOMODCACHE/golang.org/x/mod@v0.23.0/LICENSE: - -Copyright 2009 The Go Authors. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are -met: - - * Redistributions of source code must retain the above copyright -notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above -copyright notice, this list of conditions and the following disclaimer -in the documentation and/or other materials provided with the -distribution. - * Neither the name of Google LLC nor the names of its -contributors may be used to endorse or promote products derived from -this software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - - -------------------------------------------------------------------------------- Dependency : golang.org/x/oauth2 Version: v0.24.0 diff --git a/dev-tools/mage/otel/deps.go b/dev-tools/mage/otel/deps.go new file mode 100644 index 00000000000..4c949949643 --- /dev/null +++ b/dev-tools/mage/otel/deps.go @@ -0,0 +1,186 @@ +// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one +// or more contributor license agreements. Licensed under the Elastic License 2.0; +// you may not use this file except in compliance with the Elastic License 2.0. + +package otel + +import ( + "fmt" + "os" + "path/filepath" + "sort" + "strings" + + "golang.org/x/mod/modfile" + "golang.org/x/mod/module" +) + +// GetOtelDependencies returns the Otel dependencies from the given go.mod. This function applies replace directives. +func GetOtelDependencies(goModPath string) (*OtelDependencies, error) { + // read go.mod + goModBytes, err := os.ReadFile(goModPath) + if err != nil { + return nil, err + } + + goModFileName := filepath.Base(goModPath) + + modFile, err := modfile.Parse(goModFileName, goModBytes, nil) + if err != nil { + return nil, err + } + + var receivers, extensions, exporters, processors, connectors []*otelDependency + // process imports + pathToDep := make(map[string]*otelDependency) + for _, req := range modFile.Require { + dependency := newOtelDependency(req) + if dependency == nil { + continue + } + pathToDep[req.Mod.Path] = dependency + + if dependency.ComponentType == "connector" { + connectors = append(connectors, dependency) + } else if dependency.ComponentType == "exporter" { + exporters = append(exporters, dependency) + } else if dependency.ComponentType == "extension" { + extensions = append(extensions, dependency) + } else if dependency.ComponentType == "processor" { + processors = append(processors, dependency) + } else if dependency.ComponentType == "receiver" { + receivers = append(receivers, dependency) + } + } + + for _, list := range [][]*otelDependency{connectors, exporters, extensions, processors, receivers} { + sort.Slice(list, func(i, j int) bool { return list[i].Name < list[j].Name }) + } + + // take care of replaces + for _, rep := range modFile.Replace { + otelDep, ok := pathToDep[rep.Old.Path] + if ok { + otelDep.applyReplace(rep) + } + } + + return &OtelDependencies{ + Connectors: connectors, + Exporters: exporters, + Extensions: extensions, + Processors: processors, + Receivers: receivers, + }, nil +} + +type otelDependency struct { + ComponentType string + Name string + Version string + Link string + req *modfile.Require +} + +func newOtelDependency(r *modfile.Require) *otelDependency { + if !strings.Contains(r.Mod.Path, "go.opentelemetry.io/") && + !strings.Contains(r.Mod.Path, "github.com/open-telemetry/") && + !strings.Contains(r.Mod.Path, "github.com/elastic/opentelemetry-collector-components/") { + return nil + } + + if r.Indirect { + return nil + } + + componentName := getOtelComponentName(r.Mod.Path) + componentType := getOtelComponentType(r.Mod.Path) + link := getOtelDependencyLink(r.Mod.Path, r.Mod.Version) + + return &otelDependency{ + ComponentType: componentType, + Name: componentName, + Version: r.Mod.Version, + Link: link, + req: r, + } +} + +func (d *otelDependency) applyReplace(rep *modfile.Replace) { + if rep == nil || rep.Old != d.req.Mod { + return + } + d.Version = rep.New.Version + d.req.Mod = rep.New + d.Link = getOtelDependencyLink(rep.New.Path, rep.New.Version) +} + +func getOtelComponentName(dependencyName string) string { + parts := strings.Split(dependencyName, "/") + return parts[len(parts)-1] +} + +func getOtelComponentType(dependencyName string) string { + if strings.Contains(dependencyName, "/connector/") { + return "connector" + } else if strings.Contains(dependencyName, "/exporter/") { + return "exporter" + } else if strings.Contains(dependencyName, "/extension/") { + return "extension" + } else if strings.Contains(dependencyName, "/processor/") { + return "processor" + } else if strings.Contains(dependencyName, "/receiver/") { + return "receiver" + } + return "" +} + +func getOtelDependencyLink(dependencyURI string, version string) string { + dependencyRepository := getDependencyRepository(dependencyURI) + dependencyPath := strings.TrimPrefix(dependencyURI, dependencyRepository+"/") + gitRevision := fmt.Sprintf("%s/%s", dependencyPath, version) + repositoryURL := getOtelRepositoryURL(dependencyURI) + // if the version is a pseudo-version pointing to a revision without a tag, we need to extract the revision + if module.IsPseudoVersion(version) { + revision, err := module.PseudoVersionRev(version) + if err == nil { // this should never return an error, as we check it earlier + gitRevision = revision + } + } + return fmt.Sprintf("https://%s/blob/%s/%s/README.md", repositoryURL, gitRevision, dependencyPath) +} + +func getDependencyRepository(dependencyURI string) string { + dependencyURIChunks := strings.Split(dependencyURI, "/") + if len(dependencyURIChunks) < 2 { + return "" + } + var dependencyRepository string + if dependencyURIChunks[0] == "go.opentelemetry.io" { + dependencyRepository = dependencyURIChunks[0] + "/" + dependencyURIChunks[1] + } else { + dependencyRepository = dependencyURIChunks[0] + "/" + dependencyURIChunks[1] + "/" + dependencyURIChunks[2] + } + return dependencyRepository +} + +func getOtelRepositoryURL(dependencyURI string) string { + if strings.HasPrefix(dependencyURI, "go.opentelemetry.io/") { + return "github.com/open-telemetry/opentelemetry-collector" + } else if strings.HasPrefix(dependencyURI, "github.com/") { + parts := strings.SplitN(dependencyURI, "/", 4) + hostPart := parts[0] + orgPart := parts[1] + repoPart := parts[2] + return fmt.Sprintf("%s/%s/%s", hostPart, orgPart, repoPart) + } + return "" +} + +type OtelDependencies struct { + Connectors []*otelDependency + Exporters []*otelDependency + Extensions []*otelDependency + Processors []*otelDependency + Receivers []*otelDependency +} diff --git a/dev-tools/mage/otel/deps_test.go b/dev-tools/mage/otel/deps_test.go new file mode 100644 index 00000000000..3dc6ef83373 --- /dev/null +++ b/dev-tools/mage/otel/deps_test.go @@ -0,0 +1,145 @@ +// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one +// or more contributor license agreements. Licensed under the Elastic License 2.0; +// you may not use this file except in compliance with the Elastic License 2.0. + +package otel + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestGetOtelDependencies(t *testing.T) { + goModContent := `module github.com/elastic/elastic-agent + +go 1.24.1 + +require ( + github.com/elastic/opentelemetry-collector-components/connector/signaltometricsconnector v0.3.0 + github.com/open-telemetry/opentelemetry-collector-contrib/connector/routingconnector v0.119.0 + github.com/open-telemetry/opentelemetry-collector-contrib/exporter/kafkaexporter v0.119.0 + github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor v0.119.0 + github.com/open-telemetry/opentelemetry-collector-contrib/extension/healthcheckextension v0.119.0 + github.com/open-telemetry/opentelemetry-collector-contrib/pkg/status v0.119.0 + github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kafkareceiver v0.119.0 + github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.119.0 + go.opentelemetry.io/collector/component/componentstatus v0.119.0 + go.opentelemetry.io/collector/confmap/provider/envprovider v1.25.0 + go.opentelemetry.io/collector/exporter/debugexporter v0.119.0 + go.opentelemetry.io/collector/extension/memorylimiterextension v0.119.0 + go.opentelemetry.io/collector/processor/batchprocessor v0.119.0 + go.opentelemetry.io/collector/receiver/otlpreceiver v0.119.0 + golang.org/x/crypto v0.36.0 + github.com/elastic/elastic-agent-autodiscover v0.9.0 +) +require ( + cel.dev/expr v0.19.1 // indirect + github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer v0.119.0 // indirect + go.opentelemetry.io/collector v0.119.0 // indirect + go.opentelemetry.io/collector/exporter/exporterhelper/xexporterhelper v0.119.0 // indirect + go.opentelemetry.io/collector/extension/auth v0.119.0 // indirect + go.opentelemetry.io/collector/processor/processorhelper/xprocessorhelper v0.119.0 // indirect + go.opentelemetry.io/collector/receiver/xreceiver v0.119.0 // indirect +) + +replace ( + github.com/fsnotify/fsnotify => github.com/elastic/fsnotify v1.6.1-0.20240920222514-49f82bdbc9e3 + github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.119.0 => github.com/elastic/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.0.0-20250317163643-19cd4e80024f +) + +` + tempGoModFile := filepath.Join(os.TempDir(), "go.mod") + err := os.WriteFile(tempGoModFile, []byte(goModContent), 0600) + require.NoError(t, err) + t.Cleanup(func() { + removeErr := os.Remove(tempGoModFile) + assert.NoError(t, removeErr) + }) + + expected := &OtelDependencies{ + Connectors: []*otelDependency{ + { + ComponentType: "connector", + Name: "routingconnector", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/connector/routingconnector/v0.119.0/connector/routingconnector/README.md", + }, + { + ComponentType: "connector", + Name: "signaltometricsconnector", + Version: "v0.3.0", + Link: "https://github.com/elastic/opentelemetry-collector-components/blob/connector/signaltometricsconnector/v0.3.0/connector/signaltometricsconnector/README.md", + }, + }, + Exporters: []*otelDependency{ + { + ComponentType: "exporter", + Name: "debugexporter", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector/blob/exporter/debugexporter/v0.119.0/exporter/debugexporter/README.md", + }, + { + ComponentType: "exporter", + Name: "kafkaexporter", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/exporter/kafkaexporter/v0.119.0/exporter/kafkaexporter/README.md", + }, + }, + Extensions: []*otelDependency{ + { + ComponentType: "extension", + Name: "healthcheckextension", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/extension/healthcheckextension/v0.119.0/extension/healthcheckextension/README.md", + }, + { + ComponentType: "extension", + Name: "memorylimiterextension", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector/blob/extension/memorylimiterextension/v0.119.0/extension/memorylimiterextension/README.md", + }, + }, + Processors: []*otelDependency{ + { + ComponentType: "processor", + Name: "batchprocessor", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector/blob/processor/batchprocessor/v0.119.0/processor/batchprocessor/README.md", + }, + { + ComponentType: "processor", + Name: "transformprocessor", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/transformprocessor/v0.119.0/processor/transformprocessor/README.md", + }, + }, + Receivers: []*otelDependency{ + { + ComponentType: "receiver", + Name: "kafkareceiver", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/receiver/kafkareceiver/v0.119.0/receiver/kafkareceiver/README.md", + }, + { + ComponentType: "receiver", + Name: "otlpreceiver", + Version: "v0.119.0", + Link: "https://github.com/open-telemetry/opentelemetry-collector/blob/receiver/otlpreceiver/v0.119.0/receiver/otlpreceiver/README.md", + }, + { + ComponentType: "receiver", + Name: "prometheusreceiver", + Version: "v0.0.0-20250317163643-19cd4e80024f", + Link: "https://github.com/elastic/opentelemetry-collector-contrib/blob/19cd4e80024f/receiver/prometheusreceiver/README.md", + }, + }, + } + + actual, err := GetOtelDependencies(tempGoModFile) + require.NoError(t, err) + assert.EqualExportedValues(t, expected, actual) +} diff --git a/go.mod b/go.mod index f5c8a802a6e..88609bad7e6 100644 --- a/go.mod +++ b/go.mod @@ -75,9 +75,10 @@ require ( go.opentelemetry.io/collector/processor/memorylimiterprocessor v0.119.0 go.opentelemetry.io/collector/receiver/nopreceiver v0.119.0 go.uber.org/zap v1.27.0 - golang.org/x/crypto v0.36.0 + golang.org/x/crypto v0.33.0 golang.org/x/exp v0.0.0-20250215185904-eff6e970281f - golang.org/x/net v0.38.0 + golang.org/x/mod v0.23.0 + golang.org/x/net v0.35.0 golang.org/x/sync v0.12.0 golang.org/x/sys v0.31.0 golang.org/x/term v0.30.0 @@ -624,7 +625,6 @@ require ( go.starlark.net v0.0.0-20230525235612-a134d8f9ddca // indirect go.uber.org/atomic v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect - golang.org/x/mod v0.23.0 // indirect golang.org/x/oauth2 v0.24.0 // indirect golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect gonum.org/v1/gonum v0.15.1 // indirect diff --git a/go.sum b/go.sum index 1be9a009368..030ee56765a 100644 --- a/go.sum +++ b/go.sum @@ -1859,8 +1859,8 @@ golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDf golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M= golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8= golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= -golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34= -golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc= +golang.org/x/crypto v0.33.0 h1:IOBPskki6Lysi0lo9qQvbxiQ+FvsCC/YWOecCHAixus= +golang.org/x/crypto v0.33.0/go.mod h1:bVdXmD7IV/4GdElGPozy6U7lWdRXA4qyRVGJV57uQ5M= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -1955,8 +1955,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk= golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44= golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= -golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8= -golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8= +golang.org/x/net v0.35.0 h1:T5GQRQb2y08kTAByq9L4/bz8cipCdA8FbRTXewonqY8= +golang.org/x/net v0.35.0/go.mod h1:EglIi67kWsHKlRzzVMUD93VMSWGFOMSZgxFjparz1Qk= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190130055435-99b60b757ec1/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= diff --git a/magefile.go b/magefile.go index 7937b81198d..20dd078647b 100644 --- a/magefile.go +++ b/magefile.go @@ -26,13 +26,14 @@ import ( "regexp" "runtime" "slices" - "sort" "strconv" "strings" "sync" "sync/atomic" "time" + "github.com/elastic/elastic-agent/dev-tools/mage/otel" + "github.com/jedib0t/go-pretty/v6/table" "github.com/otiai10/copy" @@ -3319,7 +3320,7 @@ func (Otel) Readme() error { return fmt.Errorf("failed to parse README template: %w", err) } - data, err := getOtelDependencies() + data, err := otel.GetOtelDependencies("go.mod") if err != nil { return fmt.Errorf("Failed to get OTel dependencies: %w", err) } @@ -3341,151 +3342,6 @@ func (Otel) Readme() error { return nil } -func getOtelDependencies() (*otelDependencies, error) { - // read go.mod - readFile, err := os.Open("go.mod") - if err != nil { - return nil, err - } - defer readFile.Close() - - scanner := bufio.NewScanner(readFile) - - scanner.Split(bufio.ScanLines) - var receivers, extensions, exporters, processors, connectors []*otelDependency - // process imports - for scanner.Scan() { - l := strings.TrimSpace(scanner.Text()) - dependency := newOtelDependency(l) - if dependency == nil { - continue - } - - if dependency.ComponentType == "connector" { - connectors = append(connectors, dependency) - } else if dependency.ComponentType == "exporter" { - exporters = append(exporters, dependency) - } else if dependency.ComponentType == "extension" { - extensions = append(extensions, dependency) - } else if dependency.ComponentType == "processor" { - processors = append(processors, dependency) - } else if dependency.ComponentType == "receiver" { - receivers = append(receivers, dependency) - } - } - - for _, list := range [][]*otelDependency{connectors, exporters, extensions, processors, receivers} { - sort.Slice(list, func(i, j int) bool { return list[i].Name < list[j].Name }) - } - - return &otelDependencies{ - Connectors: connectors, - Exporters: exporters, - Extensions: extensions, - Processors: processors, - Receivers: receivers, - }, nil -} - -type otelDependency struct { - ComponentType string - Name string - Version string - Link string -} - -func newOtelDependency(l string) *otelDependency { - if !strings.Contains(l, "go.opentelemetry.io/") && - !strings.Contains(l, "github.com/open-telemetry/") && - !strings.Contains(l, "github.com/elastic/opentelemetry-collector-components/") { - return nil - } - - if strings.Contains(l, "// indirect") { - return nil - } - - chunks := strings.SplitN(l, " ", 2) - if len(chunks) != 2 { - return nil - } - dependencyURI := chunks[0] - version := chunks[1] - - componentName := getOtelComponentName(dependencyURI) - componentType := getOtelComponentType(dependencyURI) - link := getOtelDependencyLink(dependencyURI, version) - - return &otelDependency{ - ComponentType: componentType, - Name: componentName, - Version: version, - Link: link, - } -} - -func getOtelComponentName(dependencyName string) string { - parts := strings.Split(dependencyName, "/") - return parts[len(parts)-1] -} - -func getOtelComponentType(dependencyName string) string { - if strings.Contains(dependencyName, "/connector/") { - return "connector" - } else if strings.Contains(dependencyName, "/exporter/") { - return "exporter" - } else if strings.Contains(dependencyName, "/extension/") { - return "extension" - } else if strings.Contains(dependencyName, "/processor/") { - return "processor" - } else if strings.Contains(dependencyName, "/receiver/") { - return "receiver" - } - return "" -} - -func getOtelDependencyLink(dependencyURI string, version string) string { - dependencyRepository := getDependencyRepository(dependencyURI) - dependencyPath := strings.TrimPrefix(dependencyURI, dependencyRepository+"/") - repositoryURL := getOtelRepositoryURL(dependencyURI) - return fmt.Sprintf("https://%s/blob/%s/%s/%s/README.md", repositoryURL, dependencyPath, version, dependencyPath) -} - -func getDependencyRepository(dependencyURI string) string { - dependencyURIChunks := strings.Split(dependencyURI, "/") - if len(dependencyURIChunks) < 2 { - return "" - } - var dependencyRepository string - if dependencyURIChunks[0] == "go.opentelemetry.io" { - dependencyRepository = dependencyURIChunks[0] + "/" + dependencyURIChunks[1] - } else { - dependencyRepository = dependencyURIChunks[0] + "/" + dependencyURIChunks[1] + "/" + dependencyURIChunks[2] - } - return dependencyRepository -} - -func getOtelRepositoryURL(dependencyURI string) string { - if strings.HasPrefix(dependencyURI, "go.opentelemetry.io/") { - return "github.com/open-telemetry/opentelemetry-collector" - } else if strings.HasPrefix(dependencyURI, "github.com/") { - parts := strings.SplitN(dependencyURI, "/", 4) - hostPart := parts[0] - orgPart := parts[1] - repoPart := parts[2] - return fmt.Sprintf("%s/%s/%s", hostPart, orgPart, repoPart) - } - return "" -} - -type otelDependencies struct { - Connectors []*otelDependency - Exporters []*otelDependency - Extensions []*otelDependency - Processors []*otelDependency - Receivers []*otelDependency -} - type Helm mg.Namespace // RenderExamples runs the equivalent of `helm template` and `helm lint` From 84151edb7d82106b3c42df4798417656186b5f6a Mon Sep 17 00:00:00 2001 From: Shaunak Kashyap Date: Fri, 4 Apr 2025 09:29:10 -0700 Subject: [PATCH 2/3] Undoing irrelevant changes --- NOTICE.txt | 8 ++++---- go.mod | 4 ++-- go.sum | 8 ++++---- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/NOTICE.txt b/NOTICE.txt index 0c298dce444..f638e7c9765 100644 --- a/NOTICE.txt +++ b/NOTICE.txt @@ -17973,11 +17973,11 @@ THE SOFTWARE. -------------------------------------------------------------------------------- Dependency : golang.org/x/crypto -Version: v0.33.0 +Version: v0.36.0 Licence type (autodetected): BSD-3-Clause -------------------------------------------------------------------------------- -Contents of probable licence file $GOMODCACHE/golang.org/x/crypto@v0.33.0/LICENSE: +Contents of probable licence file $GOMODCACHE/golang.org/x/crypto@v0.36.0/LICENSE: Copyright 2009 The Go Authors. @@ -18084,11 +18084,11 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------------------------------------------------------------- Dependency : golang.org/x/net -Version: v0.35.0 +Version: v0.38.0 Licence type (autodetected): BSD-3-Clause -------------------------------------------------------------------------------- -Contents of probable licence file $GOMODCACHE/golang.org/x/net@v0.35.0/LICENSE: +Contents of probable licence file $GOMODCACHE/golang.org/x/net@v0.38.0/LICENSE: Copyright 2009 The Go Authors. diff --git a/go.mod b/go.mod index 88609bad7e6..6a7e6b51a3c 100644 --- a/go.mod +++ b/go.mod @@ -75,10 +75,10 @@ require ( go.opentelemetry.io/collector/processor/memorylimiterprocessor v0.119.0 go.opentelemetry.io/collector/receiver/nopreceiver v0.119.0 go.uber.org/zap v1.27.0 - golang.org/x/crypto v0.33.0 + golang.org/x/crypto v0.36.0 golang.org/x/exp v0.0.0-20250215185904-eff6e970281f golang.org/x/mod v0.23.0 - golang.org/x/net v0.35.0 + golang.org/x/net v0.38.0 golang.org/x/sync v0.12.0 golang.org/x/sys v0.31.0 golang.org/x/term v0.30.0 diff --git a/go.sum b/go.sum index 030ee56765a..1be9a009368 100644 --- a/go.sum +++ b/go.sum @@ -1859,8 +1859,8 @@ golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDf golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M= golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8= golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= -golang.org/x/crypto v0.33.0 h1:IOBPskki6Lysi0lo9qQvbxiQ+FvsCC/YWOecCHAixus= -golang.org/x/crypto v0.33.0/go.mod h1:bVdXmD7IV/4GdElGPozy6U7lWdRXA4qyRVGJV57uQ5M= +golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34= +golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -1955,8 +1955,8 @@ golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk= golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44= golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM= golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4= -golang.org/x/net v0.35.0 h1:T5GQRQb2y08kTAByq9L4/bz8cipCdA8FbRTXewonqY8= -golang.org/x/net v0.35.0/go.mod h1:EglIi67kWsHKlRzzVMUD93VMSWGFOMSZgxFjparz1Qk= +golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8= +golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190130055435-99b60b757ec1/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= From 06eb540372198aeaca9d76ac4755f6ba65b29b43 Mon Sep 17 00:00:00 2001 From: Shaunak Kashyap Date: Fri, 4 Apr 2025 10:15:12 -0700 Subject: [PATCH 3/3] Running mage update --- internal/pkg/otel/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/internal/pkg/otel/README.md b/internal/pkg/otel/README.md index 0858258af44..5c8a44321be 100644 --- a/internal/pkg/otel/README.md +++ b/internal/pkg/otel/README.md @@ -48,7 +48,6 @@ This section provides a summary of components included in the Elastic Distributi | [nopreceiver](https://github.com/open-telemetry/opentelemetry-collector/blob/receiver/nopreceiver/v0.119.0/receiver/nopreceiver/README.md) | v0.119.0 | | [otlpreceiver](https://github.com/open-telemetry/opentelemetry-collector/blob/receiver/otlpreceiver/v0.120.0/receiver/otlpreceiver/README.md) | v0.120.0 | | [prometheusreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/receiver/prometheusreceiver/v0.120.1/receiver/prometheusreceiver/README.md) | v0.120.1 | -| [prometheusreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/receiver/prometheusreceiver/=> github.com/elastic/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.0.0-20250317163643-19cd4e80024f/receiver/prometheusreceiver/README.md) | => github.com/elastic/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.0.0-20250317163643-19cd4e80024f | | [receivercreator](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/receiver/receivercreator/v0.120.1/receiver/receivercreator/README.md) | v0.120.1 | | [redisreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/receiver/redisreceiver/v0.120.1/receiver/redisreceiver/README.md) | v0.120.1 | | [zipkinreceiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/receiver/zipkinreceiver/v0.120.1/receiver/zipkinreceiver/README.md) | v0.120.1 |