Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

workflow: support generate log action #41

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 76 additions & 12 deletions src/bitdrift_public/protobuf/workflow/v1/workflow.proto
Original file line number Diff line number Diff line change
Expand Up @@ -75,15 +75,32 @@ message Workflow {
oneof rule_type {
option (validate.required) = true;
RuleLogMatch rule_log_match = 1;
RuleTimeout rule_timeout = 2;
}

reserved 2;
}

// An extra configuration for a given transition.
message TransitionExtension {
oneof extension_type {
option (validate.required) = true;
SankeyDiagramValueExtraction sankey_diagram_value_extraction = 1;
SaveTimestamp save_timestamp = 2;
SaveField save_field = 3;
}

// The timestamp of the transition will be snapped, and persisted to the workflow state. This
// can be looked up by ID by further transitions. The saved timestamp is fraction seconds since
// the Unix epoch.
message SaveTimestamp {
string id = 1;
}

// The log's field value, if available, will be snapped, and persisted to the workflow state.
// This can be looked up by ID by further transitions.
message SaveField {
string id = 1;
string field_name = 2;
}

// The configuration of a value extraction for a Sankey diagram.
Expand Down Expand Up @@ -116,15 +133,6 @@ message Workflow {
uint32 count = 2 [(validate.rules).uint32.gt = 0];
}

// Rule used to transition based on the amount of time that passed between
// when workflow run arrived at transition's "from" state and the time of
// of processing the current log.
//
// It can be used as a safety hatch to exit a workflow.
message RuleTimeout {
uint64 duration_ms = 1 [(validate.rules).uint64.gte = 0];
}

// An action to be taken when moving to a new state.
message Action {
oneof action_type {
Expand All @@ -133,6 +141,63 @@ message Workflow {
ActionEmitMetric action_emit_metric = 2;
ActionEmitSankeyDiagram action_emit_sankey_diagram = 3;
ActionTakeScreenshot action_take_screenshot = 4;
ActionGenerateLog action_generate_log = 5;
}

// Generates a new log message. This log will be injected into the workflow engine and processed
// like any other log, either by this workflow or by another workflow.
message ActionGenerateLog {
// Describes how to obtain a value used in field composition.
message ValueReference {
oneof value_reference_type {
// Fixed value that never changes.
string fixed = 1;
// The value of the field from the current log.
string field_from_current_log = 2;
// The value of a field from a previous log, saved via the SaveField extension. This is
// the ID of the extension.
string saved_field_id = 3;
// The timestamp of a previous log, saved via the SaveTimestamp extension. This is the ID
// of the extension.
string saved_timestamp_id = 4;
}
}

// A pair of values.
message ValueReferencePair {
ValueReference lhs = 1;
ValueReference rhs = 2;
}

// A field to be generated. Note that when operating on timestamps, the values are expected
// to be fractions of seconds since the Unix epoch.
message GeneratedField {
// The name of the field.
string name = 1;

// The value of the field.
oneof generated_field_value_type {
option (validate.required) = true;

// A single value.
ValueReference single = 2;

// Perform subtraction. lhs - rhs. Both values must be convertible to floating point.
ValueReferencePair subtract = 3;

// Perform addition. lhs + rhs. Both values must be convertible to floating point.
ValueReferencePair add = 4;

// Perform multiplication. lhs * rhs. Both values must be convertible to floating point.
ValueReferencePair multiply = 5;

// Perform division. lhs / rhs. Both values must be convertible to floating point.
ValueReferencePair divide = 6;
Comment on lines +185 to +195
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the two values are not divisible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another good question. IMO we have 3 options:

  1. Just drop the field entirely.
  2. Empty string for value.
  3. "NaN"

I vote 3. WDYT? @Reflejo

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same for when strings are not numbers, right? NaN makes sense assuming histograms know how to deal with that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah exactly. I will make sure the histogram code correctly drops NaN stuff.

}
}

string message = 1;
repeated GeneratedField fields = 2;
Comment on lines +199 to +200
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what would be the log level and log type of the generated log? do you want that configurable?

Copy link
Contributor Author

@mattklein123 mattklein123 Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good question. We should probably make it configurable. @Reflejo ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably let them set the log level but not the type? I think that's pretty opaque for user consumption

}

// Flush the content of the specified buffer(s) to the bitdrift control plane, with the option to continue
Expand Down Expand Up @@ -197,8 +262,7 @@ message Workflow {
// regex "[a-zA-Z_][a-zA-Z0-9_]*".
string id = 1 [(validate.rules).string = {min_len: 1}];

// The type of metric. Currently only counters are supported. In the future histograms could
// be supported as well.
// The type of metric.
oneof metric_type {
option (validate.required) = true;

Expand Down
Loading