feat(component validation): validate component_discarded_events_total for sources#17981
Closed
neuronull wants to merge 4 commits intoneuronull/draft_component_validation_better_validationfrom
Conversation
Datadog ReportBranch report: ✅ |
neuronull
commented
Jul 18, 2023
| telemetry_events: &[Event], | ||
| runner_metrics: &RunnerMetrics, | ||
| ) -> Result<Vec<String>, Vec<String>> { | ||
| let expected_dropped = runner_metrics.discarded_events_total; |
Contributor
Author
There was a problem hiding this comment.
Note- the expected value is not currently being set.
This is a rather tricky one to validate since it requires forcing the stream to close, after we sent the events to the source and before it sent them to the downstream.
tobz
reviewed
Jul 19, 2023
… into neuronull/component_validation_discarded_events
Contributor
Author
|
Because the code changed so much in #17980 , and this PR is so small, I will just close and re-implement it there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
addresses: #16842
The note in #16935 , about it being challenging to validate this metric, is true in general. That comment was specific to validating it for sources, where the events are more or less only droppable when the stream closes unexpectedly on us. Which I have no idea right now how to trigger deterministically.
Validating that with sinks should hopefully be a little easier as we could fail encoding etc.
Also a note, the code changed in this PR is altered significantly in #17980 ... posting this draft PR is mostly a tracking mechanism at this point.