Skip to content

[processor/servicegraph]speculate virtual node before edge get expired.#7

Merged
JaredTan95 merged 61 commits into
insight-mainfrom
virtual_node
Nov 24, 2022
Merged

[processor/servicegraph]speculate virtual node before edge get expired.#7
JaredTan95 merged 61 commits into
insight-mainfrom
virtual_node

Conversation

@JaredTan95

@JaredTan95 JaredTan95 commented Nov 22, 2022

Copy link
Copy Markdown

Description:

image

image

Link to tracking Issue:

Testing:

Documentation:

bogdandrutu and others added 30 commits November 11, 2022 10:50
…pen-telemetry#16258)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
…level (open-telemetry#14922)

feat: add fielddata memory size metrics on index level
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
* Rename ottllogs to ottllog

* add changelog entry

* fix lint
Moving dependency to version in main to move open-telemetry/opentelemetry-collector#6511 in core forward.
This test fails periodially with a non-zero exit code returned from
the setup script. The likely cause is that the wait strategy was not
robust enough, in that the rabbitmqadmin command was not ready before
associated commands were run.

This fix incorporates the setup script into a custom wait strategy,
which allows the script to be rerun until it succeeds.
…ry#16300)

dependabot updates Tue Nov 15 00:17:54 UTC 2022
Bump github.com/aerospike/aerospike-client-go/v6 from 6.4.0 to 6.6.0 in /receiver/aerospikereceiver
Bump github.com/aws/aws-sdk-go-v2/config from 1.17.10 to 1.18.0 in /extension/sigv4authextension
Bump go.mongodb.org/atlas from 0.18.0 to 0.19.0 in /receiver/mongodbatlasreceiver
Bump google.golang.org/api from 0.102.0 to 0.103.0 in /receiver/googlecloudspannerreceiver
Bump k8s.io/api from 0.25.3 to 0.25.4 in /processor/k8sattributesprocessor
Bump k8s.io/apimachinery from 0.25.3 to 0.25.4 in /internal/k8sconfig
Bump k8s.io/apimachinery from 0.25.3 to 0.25.4 in /processor/resourcedetectionprocessor
Bump k8s.io/client-go from 0.25.3 to 0.25.4 in /processor/k8sattributesprocessor

Co-authored-by: codeboten <codeboten@users.noreply.github.com>
The container requests where using same fixed, exposed port, instead of
using the random one. In this commit we are calculating the exposed port
in the getContainer method, returning the endpoint for the container
in order to be used by each test
…telemetry#16280)

* fix: fix port feature gate's description

* remove changelog
…#16264)

* chore: use testcontainers-go in zookeper receiver

* chore: use testcontainers-go in memcached receiver

* chore: use testcontainers-go in docker stats receiver

* chore: use testcontainers-go in aerospike receiver

* chore: use testcontainers-go in kafka metrics receiver

* chore: use testcontainers-go in docker observer extension
…s for Redis versions >= 6.0 (open-telemetry#14943)

Update Redis Receiver to Support CPU Metrics After 6.0

Add more state values to redis.cpu.time metrics to support sys_main_thread and user_main_thread cpu usage time.
…lemetry#16301)

The methods were deprecated since there is no establish convention on how to decode empty trace or span IDs at this point. The current method behavior is preserved in functions added in internal/coreinternal/traceutil. This commit updates all the usages of HexString where the result is used as identifier to the new internal methods.
…pen-telemetry#16302)

Avoid unnecessary string encoding and decoding of trace ID. Create UUID from bytes directly
…n-telemetry#16298)

* Commonize logic for handling slice and non-slice args

* Add PR number for issue reference

Co-authored-by: Evan Bradley <evan-bradley@users.noreply.github.com>
1. In adoption to deprecation of [Span|Trace]ID.HexString method, this change includes migration to [Span|Trace]ID implementing fmt.Stringer interface for display purposes.
2. Adopt to service.State enum values renames
It's not used anymore, in favor of testcontainers-go
…ding (open-telemetry#16322)

Make flaky test less demanding, and add more checks to make sure the test actually tests a behavior of the exporter
…trics (open-telemetry#16324)

[exporter/splunkhecexporter] Do not log a warning on mapping empty metrics.
* fix demo could not stat up

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

* remove unnecessary GOOS=linux

Signed-off-by: Jared Tan <jian.tan@daocloud.io>

Signed-off-by: Jared Tan <jian.tan@daocloud.io>
* add a new method BuildWithSplitFunc, this method can directly pass in a user-defined splitFunc
aboguszewski-sumo and others added 20 commits November 17, 2022 16:34
…rd types (open-telemetry#14875)

* feat: add cluster health metrics for two more shard types
…#16362)


Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
These tests needed a little love to make it easier to contribute to the
component

Co-authored-by: Alex Boten <alex@boten.ca>
Add new approvers to remaining lists

Co-authored-by: Evan Bradley <evan-bradley@users.noreply.github.com>
…t pool (open-telemetry#16250)

Adding new feature to allow the user to add their own CA to the ingest and api client cert pool.
This is needed when the exporter is pointing to a TLS enabled signalfx receiver or/and TLS enabled http_forwarder
and the CA is not in the system cert pool (ex: agent <--> GW setup)

Signed-off-by: Dani Louca <dlouca@splunk.com>
Co-authored-by: Dmitrii Anoshin <anoshindx@gmail.com>
…and operation time metrics (open-telemetry#16170)

* feat: add missing operation count datapoints with total aggregation

* feat: add missing operation count datapoints with primary shards aggregation

* feat: add missing operation time datapoints with total shard aggregation

* feat: add missing operation time datapoints with primary shards aggregations
* Update the jmx-metrics version to

* Update add-jmx-metrics-gatherer-1.20.0-alpha.yaml

Co-authored-by: Alex Boten <alex@boten.ca>
…etry#16354)

* [exporter/azuremonitor] Support span and exception events

Exception events flow to the AppInsights Exception table
Span events flow to the Traces table, when enabled.

* Add changelog entry

* Exporting span events is off by default
…c name (open-telemetry#16071)

 Trim type's and unit's suffixes from metric name as per otel specs.

There was a lot of changes required to pass down a featuregate.Registry in order to be testable. The important bit in the receiver code path is the metricFamily.appendMetric which trims the suffixes when creating an OTLP metric.

Units suffixes will only take effect when using the OpenMetrics format, since the prometheus parser doesn't support units.

This is being guarded by the gate: --feature-gates=pkg.translator.prometheus.NormalizeName
 Adds support for setting maps in Values. The primary impact is the ability for contexts to set maps as attribute values.
We used only one constant for error message from "github.com/jaegertracing/jaeger/cmd/collector/app/handler"
…open-telemetry#16372)

Return full error instead of format string "Unable to process request body: %v"
Signed-off-by: Alex Boten <aboten@lightstep.com>

Signed-off-by: Alex Boten <aboten@lightstep.com>
…emetry#16376)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
@JaredTan95 JaredTan95 changed the base branch from main to insight-main November 22, 2022 07:25
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
Comment thread processor/servicegraphprocessor/internal/store/store.go Outdated
Signed-off-by: Jared Tan <jian.tan@daocloud.io>
@JaredTan95 JaredTan95 merged this pull request into insight-main Nov 24, 2022
@JaredTan95 JaredTan95 deleted the virtual_node branch November 24, 2022 03:06
JaredTan95 pushed a commit that referenced this pull request Oct 4, 2024
… Histo --> Histogram (open-telemetry#33824)

## Description

This PR adds a custom metric function to the transformprocessor to
convert exponential histograms to explicit histograms.

Link to tracking issue: Resolves open-telemetry#33827

**Function Name**
```
convert_exponential_histogram_to_explicit_histogram
```

**Arguments:**

- `distribution` (_upper, midpoint, uniform, random_)
- `ExplicitBoundaries: []float64`

**Usage example:**

```yaml
processors:
  transform:
    error_mode: propagate
    metric_statements:
    - context: metric
      statements:
        - convert_exponential_histogram_to_explicit_histogram("random", [10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]) 
```

**Converts:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: ExponentialHistogram
     -> AggregationTemporality: Delta
ExponentialHistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-31 09:35:25.212037 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
Bucket (32.000000, 64.000000], Count: 10
Bucket (64.000000, 128.000000], Count: 22
Bucket (128.000000, 256.000000], Count: 12
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

**To:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: Histogram
     -> AggregationTemporality: Delta
HistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-30 21:37:07.830902 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
ExplicitBounds #0: 10.000000
ExplicitBounds #1: 20.000000
ExplicitBounds #2: 30.000000
ExplicitBounds #3: 40.000000
ExplicitBounds #4: 50.000000
ExplicitBounds #5: 60.000000
ExplicitBounds #6: 70.000000
ExplicitBounds #7: 80.000000
ExplicitBounds #8: 90.000000
ExplicitBounds #9: 100.000000
Buckets #0, Count: 0
Buckets #1, Count: 0
Buckets #2, Count: 0
Buckets #3, Count: 2
Buckets #4, Count: 5
Buckets #5, Count: 0
Buckets #6, Count: 3
Buckets #7, Count: 7
Buckets #8, Count: 2
Buckets #9, Count: 4
Buckets #10, Count: 21
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

### Testing

- Several unit tests have been created. We have also tested by ingesting
and converting exponential histograms from the `statsdreceiver` as well
as directly via the `otlpreceiver` over grpc over several hours with a
large amount of data.

- We have clients that have been running this solution in production for
a number of weeks.

### Readme description:

### convert_exponential_hist_to_explicit_hist

`convert_exponential_hist_to_explicit_hist([ExplicitBounds])`

the `convert_exponential_hist_to_explicit_hist` function converts an
ExponentialHistogram to an Explicit (_normal_) Histogram.

`ExplicitBounds` is represents the list of bucket boundaries for the new
histogram. This argument is __required__ and __cannot be empty__.

__WARNING:__

The process of converting an ExponentialHistogram to an Explicit
Histogram is not perfect and may result in a loss of precision. It is
important to define an appropriate set of bucket boundaries to minimize
this loss. For example, selecting Boundaries that are too high or too
low may result histogram buckets that are too wide or too narrow,
respectively.

---------

Co-authored-by: Kent Quirk <kentquirk@gmail.com>
Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com>
JaredTan95 pushed a commit that referenced this pull request Apr 11, 2025
…ry#38146)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description

This PR adds support for configuring metric temporality (delta or
cumulative) in the telemetrygen tool. This allows users to generate
metrics with different temporality types, which is particularly useful
for testing different metric collection scenarios.

##### Changes

- Added `temporalityType` flag for metrics pipeline that accepts values:
`delta` or `cumulative`


<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#38073 

<!--Describe what testing was performed and which tests were added.-->
#### Testing

Testing was performed by setting up as simple collector with `otlp
`receiver and `debug` exporter

##### Sum Datapoint

```
Resource SchemaURL: https://opentelemetry.io/schemas/1.13.0
Resource attributes:
     -> service.name: Str(telemetrygen)
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: gen
     -> Description: 
     -> Unit: 
     -> DataType: Histogram
     -> AggregationTemporality: Delta
```

##### Histogram Datapoint

```
Count: 0
Sum: 3940.000000
ExplicitBounds #0: 0.000000
ExplicitBounds #1: 5.000000
ExplicitBounds #2: 10.000000
ExplicitBounds #3: 25.000000
ExplicitBounds #4: 50.000000
ExplicitBounds #5: 75.000000
ExplicitBounds #6: 100.000000
ExplicitBounds #7: 250.000000
ExplicitBounds #8: 500.000000
ExplicitBounds #9: 750.000000
ExplicitBounds #10: 1000.000000
ExplicitBounds #11: 2500.000000
ExplicitBounds #12: 5000.000000
ExplicitBounds #13: 7500.000000
ExplicitBounds #14: 10000.000000
Buckets #0, Count: 0
Buckets #1, Count: 0
Buckets #2, Count: 1
Buckets #3, Count: 0
Buckets #4, Count: 0
Buckets #5, Count: 0
Buckets #6, Count: 3
Buckets #7, Count: 4
Buckets #8, Count: 1
Buckets #9, Count: 1
Buckets #10, Count: 0
Buckets #11, Count: 0
Buckets #12, Count: 0
Buckets #13, Count: 0
Buckets #14, Count: 0
```


- [x] Need to add tests for the config

---------

Signed-off-by: Nikos Angelopoulos <nikolaos.angelopoulos@grafana.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.