Demo notebook to log and create model serving service for a MedGemma model #154
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Pull Request Overview
This pull request adds a demo notebook and documentation for logging and serving LLaMA models using Snowpark Container Services (SPCS). The demo provides an end-to-end example showing how to create a transformer pipeline model, log it to the Snowflake ML registry, and deploy it as a service for inference.
- Adds comprehensive documentation for Model Serving setup and prerequisites
- Creates a complete Jupyter notebook demonstrating LLaMA model deployment workflow
- Includes examples for both programmatic model inference and REST API calls
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| LLM Model Serving/README.md | Provides setup instructions, prerequisites, and references to the demo notebook |
| LLM Model Serving/LLAMA_MODEL_QUICKSTART.ipynb | Complete notebook demonstrating LLaMA model logging, service creation, and inference |
LLM Model Serving/README.md
Outdated
| 1. Create a Compute Pool [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-a-compute-pool) | ||
| 2. Create a Image Repository [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) |
There was a problem hiding this comment.
The markdown link text should be descriptive rather than generic. Replace '[link]' with descriptive text like '[Snowflake Documentation]' for better accessibility and clarity.
| 1. Create a Compute Pool [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-a-compute-pool) | |
| 2. Create a Image Repository [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) | |
| 1. [Create a Compute Pool](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-a-compute-pool) | |
| 2. [Create an Image Repository](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) |
Copilot uses AI. Check for mistakes.
LLM Model Serving/README.md
Outdated
| ### Prerequisites | ||
|
|
||
| 1. Create a Compute Pool [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-a-compute-pool) | ||
| 2. Create a Image Repository [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) |
There was a problem hiding this comment.
The markdown link text should be descriptive rather than generic. Replace '[link]' with descriptive text like '[Snowflake Documentation]' for better accessibility and clarity.
| 2. Create a Image Repository [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) | |
| 2. Create an Image Repository [Create an Image Repository](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) |
Copilot uses AI. Check for mistakes.
LLM Model Serving/README.md
Outdated
| ### Prerequisites | ||
|
|
||
| 1. Create a Compute Pool [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-a-compute-pool) | ||
| 2. Create a Image Repository [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) |
There was a problem hiding this comment.
Grammar error: should be 'an Image Repository' instead of 'a Image Repository'.
| 2. Create a Image Repository [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) | |
| 2. Create an Image Repository [link](https://docs.snowflake.com/en/developer-guide/snowflake-ml/model-registry/container#create-an-image-repository) |
Copilot uses AI. Check for mistakes.
| " # TODO: Add your image repo here\n", | ||
| " image_repo=\"<your-image-repo>\",\n", | ||
| " # TODO: Add your compute pool here\n", | ||
| " service_compute_pool=\"<your-compute-pool>\",\n", |
There was a problem hiding this comment.
The placeholder values should include more specific guidance about the expected format or provide example values to help users understand what to substitute.
| " # TODO: Add your image repo here\n", | |
| " image_repo=\"<your-image-repo>\",\n", | |
| " # TODO: Add your compute pool here\n", | |
| " service_compute_pool=\"<your-compute-pool>\",\n", | |
| " # TODO: Add your image repo here (e.g., \"orgname/repo:tag\" or \"123456789012.dkr.ecr.us-west-2.amazonaws.com/my-llama-image:latest\")\n", | |
| " image_repo=\"123456789012.dkr.ecr.us-west-2.amazonaws.com/my-llama-image:latest\",\n", | |
| " # TODO: Add your compute pool here (e.g., \"MY_COMPUTE_POOL\")\n", | |
| " service_compute_pool=\"MY_COMPUTE_POOL\",\n", |
Copilot uses AI. Check for mistakes.
| "outputs": [], | ||
| "source": [ | ||
| "# List all services in a compute pool\n", | ||
| "session.sql(\"SHOW SERVICES IN COMPUTE POOL <your-compute-pool>\").collect()" |
There was a problem hiding this comment.
The placeholder '' should be replaced with a more descriptive example or variable reference to maintain consistency with the service creation code above.
| "session.sql(\"SHOW SERVICES IN COMPUTE POOL <your-compute-pool>\").collect()" | |
| "session.sql(f\"SHOW SERVICES IN COMPUTE POOL {compute_pool_name}\").collect()" |
Copilot uses AI. Check for mistakes.
| "source": [ | ||
| "# TODO: change the `url` to the service ingress URL\n", | ||
| "# this can be found in the \"SHOW ENDPOINTS IN SERVICE LLAMA_3_SERVICE\" sql query output above\n", | ||
| "url = \"http://{ingress-url}/--call--\"\n", |
There was a problem hiding this comment.
[nitpick] The placeholder format is inconsistent with other placeholders in the notebook. Consider using '' format for consistency with other placeholder patterns.
| "url = \"http://{ingress-url}/--call--\"\n", | |
| "url = \"http://<ingress-url>/--call--\"\n", |
Copilot uses AI. Check for mistakes.
| " model=llama_3_model,\n", | ||
| " model_name=\"llama_3\",\n", | ||
| " version_name=\"V1\",\n", | ||
| " target_platforms=[\"SNOWPARK_CONTAINER_SERVICES\"],\n", |
There was a problem hiding this comment.
use SNOWPARK_CONTAINER_SERVICES_ONLY
| "mv.create_service(\n", | ||
| " service_name=\"llama_3_service\",\n", | ||
| " # TODO: Add your image repo here\n", | ||
| " image_repo=\"<your-image-repo>\",\n", |
There was a problem hiding this comment.
create separate variable for COMPUTE_POOL_NAME and IMAGE_REPO_NAME in separate cell and use everywhere
| "outputs": [], | ||
| "source": [ | ||
| "# List all services in a compute pool\n", | ||
| "session.sql(\"SHOW SERVICES IN COMPUTE POOL <your-compute-pool>\").collect()" |
There was a problem hiding this comment.
can you do mv.list_service() instead?
| "outputs": [], | ||
| "source": [ | ||
| "# List all endpoints in a service\n", | ||
| "session.sql(\"SHOW ENDPOINTS IN SERVICE LLAMA_3_SERVICE\").collect()" |
| "source": [ | ||
| "output_df = mv.run(\n", | ||
| " X=x_df,\n", | ||
| " function_name=\"__call__\",\n", |
There was a problem hiding this comment.
is this needed even when we have only one function?
| "def initiate_snowflake_connection():\n", | ||
| " connection_parameters = SnowflakeLoginOptions()\n", | ||
| " connection_parameters[\"session_parameters\"] = {\n", | ||
| " \"PYTHON_CONNECTOR_QUERY_RESULT_FORMAT\": \"json\"\n", |
There was a problem hiding this comment.
this will be disallowed soon. Please use PAT token
| "source": [ | ||
| "# TODO: change the `url` to the service ingress URL\n", | ||
| "# this can be found in the \"SHOW ENDPOINTS IN SERVICE LLAMA_3_SERVICE\" sql query output above\n", | ||
| "url = \"http://{ingress-url}/--call--\"\n", |
There was a problem hiding this comment.
can't we auto-generate based on output for list_services()?
| "source": [ | ||
| "service_name = \"llama_3_service\"\n", | ||
| "# TODO: Add your image repo here\n", | ||
| "image_repo = \"<your-image-repo>\"\n", |
| "outputs": [], | ||
| "source": [ | ||
| "def get_headers():\n", | ||
| " # TODO: change the `pat_token` to your PAT token\n", |
There was a problem hiding this comment.
Can you put instructions on how to get the PAT token in the above markdown cell?
sfc-gh-pramachandran
changed the title
No description provided.