How does Prefect server run Python code that relies on external packages?

[wigging]

I have a basic flow example that imports the NumPy package as shown below. This code resides on my local machine in a uv project where I define the local dependencies.

# example.py

import numpy as np
from prefect import flow

@flow(log_prints=True)
def hello(name: str = "Marvin"):
    print("Hello", name)

    a = np.array([1, 2, 3, 4, 5])
    print("Array is", a)

if __name__ == "__main__":
    hello("Homer Simpson")

Before running the code, I set the Prefect API URL environment variable to point to the Prefect server running on a separate computer on my network:

export PREFECT_API_URL="http://x.x.x.x:4200/api"

Next, I run the Python file with uv as follows:

$ uv run example.py

In the Prefect dashboard on my other computer, I see the output from the print statements on the run page. But the environment on the other computer where the Prefect server is running does not have the NumPy package. So how is the Prefect server running this code? Is the server automatically installing the NumPy package before it runs the Python code? Or does the Python code not actually run on the server machine?

[zzstoatzz]

hi @wigging

Or does the Python code not actually run on the server machine?

correct, the server is not the place where your code runs.

your code runs "client-side" someplace, which could be

• your other computer
• an EC2 machine
• a kubernetes pod

that client-side place just needs to have PREFECT_API_URL set appropriately (and PREFECT_API_KEY if you use cloud) and it will make the api calls to tell the server about the run (state changes etc). the server just keeps track of things so that clients (the SDK, the UI) can ask questions like "what is the current state of flow run abcd?"

in prefect we have a concept of deployments so that you can control all the details of that client side place your code runs via work pools

for example, you might build a custom image that includes numpy and set that as a job_variable on your deployment so that when it runs, it takes your base config from your work pool and additionally specify that your flow run should use the custom image with your deps

[wigging]

Thanks for the explanation. I was thinking of the Prefect server as a place to send Python code for remote execution. But it sounds like deploying to work pools is what I'm looking for. So basically I would use the Prefect server to deploy some code to a work pool where it can be executed. Am I understanding this correctly?

[zzstoatzz]

here's a component diagram i've shared before

deploy some code to a work pool

broadly yea but just to be pedantic, a work pool is just a server-side representation of a place where code can run. a worker is the daemon that submits scheduled flow runs according to the work pool's "base job template"

ie

• KubernetesWorker submits flow runs scheduled in a k8s work pool as individual k8s pods
• ProcessWorker submits flow runs scheduled in a process work pool as subprocesses of the prefect worker start process

both work pool types have an env field where you could set ENV=DEV as an environment variable on the resulting flow run infra, but each work pool has its own unique config options in addition to the shared ones like env