Skip to content

fix(schemas): Dont panic with non object field kinds #17140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
StephenWakely merged 26 commits into from
Apr 27, 2023
Merged

fix(schemas): Dont panic with non object field kinds #17140

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
3ba9721
Dont panic with non object field kinds
StephenWakely Apr 13, 2023
0705a76
Check remap input definition is never
StephenWakely Apr 13, 2023
4123f82
Spelling
StephenWakely Apr 13, 2023
0e86124
Add enrichment tables to the transform outputs
StephenWakely Apr 17, 2023
ef60749
Handle remap array outputs
StephenWakely Apr 18, 2023
8ed6528
Return the panics
StephenWakely Apr 19, 2023
55ddb8d
Revert "Return the panics"
StephenWakely Apr 19, 2023
bd50d03
Return the panics
StephenWakely Apr 19, 2023
516cb30
Added multiple transform tests
StephenWakely Apr 19, 2023
4ad079f
Mild spacing
StephenWakely Apr 19, 2023
1e7c02b
Mild spacing
StephenWakely Apr 20, 2023
a0fd678
Add transport_output_ids to wrap calls to output that don't need defi…
StephenWakely Apr 20, 2023
5e46f19
Test even a VRL error results in the correct ports
StephenWakely Apr 20, 2023
5ec3cfd
Test a mix of array and non array results
StephenWakely Apr 21, 2023
84db500
Error if a returned definition contains never
StephenWakely Apr 24, 2023
252f3a7
Remove never check in transform, the framework should handle it
StephenWakely Apr 24, 2023
68defbf
Feedback from Kyle
StephenWakely Apr 25, 2023
5d7a8b2
Feedback from Kyle
StephenWakely Apr 25, 2023
8eaaa85
Feedback from Spencer
StephenWakely Apr 26, 2023
2e8ac39
Spelling
StephenWakely Apr 26, 2023
8214abc
Feedback from Nathan
StephenWakely Apr 27, 2023
3e0bb2b
Whitespace
StephenWakely Apr 27, 2023
f472c07
Only use transform definition fields when schema is enabled
StephenWakely Apr 27, 2023
925f413
Fixed syntax error
StephenWakely Apr 27, 2023
dced80a
Clippy
StephenWakely Apr 27, 2023
569994a
Fixed test. Non object top level fields need removing
StephenWakely Apr 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 12 additions & 10 deletions lib/vector-core/src/schema/definition.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,8 +265,9 @@ impl Definition {
/// A non-root required field means the root type must be an object, so the type will be automatically
/// restricted to an object.
///
/// Raises a warning if the path is not root, and the definition does not allow the type to be an object.
///
Comment thread
StephenWakely marked this conversation as resolved.
Outdated
/// # Panics
/// - If the path is not root, and the definition does not allow the type to be an object.
/// - Provided path has one or more coalesced segments (e.g. `.(foo | bar)`).
#[must_use]
pub fn with_event_field(
Expand All @@ -275,10 +276,10 @@ impl Definition {
kind: Kind,
meaning: Option<&str>,
) -> Self {
if !path.is_root() {
assert!(
self.event_kind.as_object().is_some(),
"Setting a field on a value that cannot be an object"
if !path.is_root() && self.event_kind.as_object().is_none() {
warn!(
"Setting a field at `{}` on a value that cannot be an object",
path.to_string()
);
}

Expand Down Expand Up @@ -327,8 +328,9 @@ impl Definition {
/// A non-root required field means the root type must be an object, so the type will be automatically
/// restricted to an object.
///
/// Raises a warning if the path is not root, and the definition does not allow the type to be an object.
///
Comment thread
StephenWakely marked this conversation as resolved.
Outdated
/// # Panics
/// - If the path is not root, and the definition does not allow the type to be an object
/// - Provided path has one or more coalesced segments (e.g. `.(foo | bar)`).
#[must_use]
pub fn with_metadata_field(
Expand All @@ -337,10 +339,10 @@ impl Definition {
kind: Kind,
meaning: Option<&str>,
) -> Self {
if !path.is_root() {
assert!(
self.metadata_kind.as_object().is_some(),
"Setting a field on a value that cannot be an object"
if !path.is_root() && self.metadata_kind.as_object().is_none() {
warn!(
"Setting a field at `{}` on a value that cannot be an object",
path.to_string()
);
}

Expand Down
14 changes: 14 additions & 0 deletions src/transforms/remap.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,20 @@ impl TransformConfig for RemapConfig {
let mut default_definitions = HashMap::new();

for (output_id, input_definition) in input_definitions {
if input_definition.event_kind().is_never() {
// If the input definition is `never` this means that the
// upstream component has errored whilst loading (for example,
// with a VRL compiler error) this means data will never be
// recieved from that component.
Comment thread Fixed
//
// This will most likely stop Vector from running, but we
// want to continue compiling to retrieve diagnostics. We
// pass on the `never` definition downstream.
default_definitions.insert(output_id.clone(), input_definition.clone());
dropped_definitions.insert(output_id.clone(), input_definition.clone());
continue;
}

let default_definition = compiled
.clone()
.map(|(state, meaning)| {
Expand Down