Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to select and invoke flow runs with Blocks when using Block as a param type hint #14393

Open
4 tasks done
jcbiddle opened this issue Jun 28, 2024 · 3 comments
Open
4 tasks done
Labels
3.x bug Something isn't working

Comments

@jcbiddle
Copy link

jcbiddle commented Jun 28, 2024

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I searched the Prefect documentation for this issue.
  • I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

I have a flow that uses different blocks to connect to different database endpoints and want the block name to be a parameter of the flow. If I use a string parameter for the block name and load the existing block from this, the code works as expected. However, to provide additional clarity I want the Prefect UI to give a dropdown list of existing blocks, rather than require the user to recall the string identifier.

I found that adding the block class as a type hint causes the Prefect UI to do exactly what I want, giving a dropdown menu for existing blocks of the correct type, with functionality to add a new block. However, attempting to execute the run after selecting a block doesn't work. No error is provided in the form input view, however, in the JSON view it is clear that the parameter JSON expects the block parameters to be provided, even if the block is a reference to an existing block with the parameters pre-configured.

To replicate this issue using the MWE provided, first register CustomBlock with:
prefect block register --file mwe.py
Then load the existing block and deploy the flow with
python mwe.py

The run deployment menu appears as
image
which seems to be exactly the desired behaviour. However, if we run the flow from the JSON parameter screen, we get the following error:
image

I assume the existing block parameters are supposed to be loaded into the run parameters, but this doesn't seem to be working as intended.

As a side-note, there also seems to be a bug where if the user switches to the JSON parameter view, the button to switch back to the form view doesn't work and requires a page refresh instead.

Reproduction

from prefect import flow
from prefect.blocks.core import Block


class CustomBlock(Block):
    example_field: str


@flow
def my_flow(block: CustomBlock):
    b = CustomBlock.load(block)
    print(b.example_field)


if __name__ == "__main__":
    my_block = CustomBlock(example_field="hello world")
    my_block.save("my-block")

    my_flow.serve("flow-deployment")

Error

- block
- example_field
- 'example_field' is a required property

Versions (prefect version output)

Version:             3.0.0rc9
API version:         0.8.4
Python version:      3.11.9
Git commit:          b75631a7
Built:               Mon, Jun 24, 2024 5:50 PM
OS/Arch:             linux/x86_64
Profile:             default
Server type:         ephemeral
Pydantic version:    2.7.4
Server:
  Database:          sqlite
  SQLite version:    3.37.2

Additional context

No response

@jcbiddle jcbiddle added 3.x bug Something isn't working needs:triage Needs feedback from the Prefect product team labels Jun 28, 2024
@WillRaphaelson
Copy link
Contributor

Thanks for the issue @jcbiddle, we can look at supporting this in the near future. Variables should work as you expect here in terms of the dropdown user experience, and then you can use the variable value in your block.load() call.

@zhen0 zhen0 removed the needs:triage Needs feedback from the Prefect product team label Jun 29, 2024
@jcbiddle
Copy link
Author

Thanks for having a look at this @WillRaphaelson. I'm new to Prefect, can you possibly expand on how I can use variables in this context to get a dropdown menu in the UI? I tried adding Variable as a type hint for the flow parameter, but that just gave me the option to make a new variable rather than select an existing one.

@WillRaphaelson
Copy link
Contributor

Hey @jcbiddle, for sure.

The type hint can be anything, in this case I lets use str. When you go to run the flow from the deployment, hit the three dots and use select variable. Then you'll get your dropdown. The value of the variable will then fall through to your flow run, and you can use that to retrieve the appropriate block with block.load(variable_value). This does require you maintain a list of variables in congruence with your blocks (or at least the ones that you want available for use in the flow). this is a workaround, and im going to keep this issue open with an edited title to reflect the feature request of allowing users to select blocks as a dropdown and invoke flows with them without erroring out :)

image

@WillRaphaelson WillRaphaelson changed the title Type hinting with Block types causes incorrect data validation error Allow users to select and invoke flow runs with Blocks when using Block as a param type hint Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.x bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants