Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VAI Provider - Run completion Error Cycle on missing labels #427

Open
grahamia opened this issue Jan 6, 2025 · 0 comments
Open

VAI Provider - Run completion Error Cycle on missing labels #427

grahamia opened this issue Jan 6, 2025 · 0 comments

Comments

@grahamia
Copy link
Contributor

grahamia commented Jan 6, 2025

Is your feature request related to a problem? Please describe.
If a run in Vertex AI is manually created and misses the required labels of runconfiguration-name or run-name the completion event in the VAI provider is created and sent to the webhook of the KFP Operator with nil for both RunConfigurationName and RunName in the run completion event data which the operator then rejects returning a 400 status error.

Describe the solution you'd like
The VAI provider should validate the request before sending to KFP Operator.
The following could be validated as mandatory:

  • Status
  • PipelineName
  • RunConfigurationName or RunName
  • RunId
  • Provider

The event should be logged and acked off pubsub to clear the broken event. This will be reported in metrics covered by #421

Describe alternatives you've considered
Could do nothing as event will expire off pubsub, it doesn't break anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant