diff --git a/docs/customizer/models/embedding.mdx b/docs/customizer/models/embedding.mdx index f22bb5e5a2..14ea7c522d 100644 --- a/docs/customizer/models/embedding.mdx +++ b/docs/customizer/models/embedding.mdx @@ -42,7 +42,7 @@ Create a Model Entity for this embedding model: -Embedding models only support **merged LoRA** (`peft` with `merge=True`). Unmerged LoRA adapters are not supported because the embedding NIM requires ONNX format, which cannot represent standalone adapters. +Embedding models only support **merged LoRA** by setting `training.finetuning_type` to `"lora_merged"` in the Automodel job input. Unmerged LoRA adapters are not supported because the embedding NIM requires ONNX format, which cannot represent standalone adapters. ### Resource Requirements diff --git a/docs/customizer/tutorials/embedding-customization-job.ipynb b/docs/customizer/tutorials/embedding-customization-job.ipynb index 7071636c51..ee2332e9f4 100644 --- a/docs/customizer/tutorials/embedding-customization-job.ipynb +++ b/docs/customizer/tutorials/embedding-customization-job.ipynb @@ -449,7 +449,7 @@ "\n", "#### Quick Setup Example\n", "\n", - "This tutorial fine-tunes [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2), NVIDIA embedding model optimized for question-answering and retrieval tasks." + "This tutorial fine-tunes [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2), an NVIDIA embedding model optimized for question-answering and retrieval tasks." ] }, { @@ -495,7 +495,7 @@ "source": [ "### 7. Create Base Model FileSet and Model Entity\n", "\n", - "Create a fileset pointing to the [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time." + "Create a fileset pointing to the [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time." ] }, { @@ -513,7 +513,7 @@ " base_model_fs = client.files.filesets.create(\n", " workspace=\"default\",\n", " name=MODEL_NAME,\n", - " description=\"NVIDIA Llama 3.2 NV EmbedQA 1B v2 embedding model\",\n", + " description=\"NVIDIA Llama Nemotron Embed 1B v2 embedding model\",\n", " storage=HuggingfaceStorageConfigParam(\n", " type=\"huggingface\",\n", " # repo_id is the full model name from Hugging Face\n", @@ -986,4 +986,4 @@ }, "nbformat": 4, "nbformat_minor": 2 -} \ No newline at end of file +} diff --git a/docs/customizer/tutorials/embedding-customization-job.mdx b/docs/customizer/tutorials/embedding-customization-job.mdx index 83bfe8896f..04dceb001b 100644 --- a/docs/customizer/tutorials/embedding-customization-job.mdx +++ b/docs/customizer/tutorials/embedding-customization-job.mdx @@ -368,7 +368,7 @@ Get your credentials: #### Quick Setup Example -This tutorial fine-tunes [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2), NVIDIA embedding model optimized for question-answering and retrieval tasks. +This tutorial fine-tunes [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2), an NVIDIA embedding model optimized for question-answering and retrieval tasks. ```python # Create secrets for model access @@ -404,7 +404,7 @@ print(f"NGC_API_KEY secret: {NGC_SECRET_NAME}") ### 7. Create Base Model FileSet and Model Entity -Create a fileset pointing to the [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time. +Create a fileset pointing to the [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time. ```python import time @@ -418,7 +418,7 @@ try: base_model_fs = client.files.filesets.create( workspace="default", name=MODEL_NAME, - description="NVIDIA Llama 3.2 NV EmbedQA 1B v2 embedding model", + description="NVIDIA Llama Nemotron Embed 1B v2 embedding model", storage=HuggingfaceStorageConfigParam( type="huggingface", # repo_id is the full model name from Hugging Face diff --git a/docs/customizer/tutorials/import-hf-model.mdx b/docs/customizer/tutorials/import-hf-model.mdx index 6f89cbddf5..8160f4a531 100644 --- a/docs/customizer/tutorials/import-hf-model.mdx +++ b/docs/customizer/tutorials/import-hf-model.mdx @@ -9,20 +9,21 @@ Use this tutorial to learn how to import a private HuggingFace model into NeMo C ## Prerequisites + ### Tutorial-Specific Prerequisites -- Access to Data Store and Deployment Manager service -- `hf` cli installed on a machine with internet access [(installation instructions)](https://huggingface.co/docs/huggingface_hub/main/en/guides/cli). -- A HuggingFace model with a **compatible architecture**. Not all HuggingFace models are compatible with NeMo Customizer. This tutorial uses `gemma-2-2b-it` as an example, but success depends on architectural compatibility. -- A HuggingFace API token and proper authentication setup. -- Sufficient storage space for the model files (typically 5-50GB depending on model size) -- At least 8GB GPU memory for smaller models, more for larger models +- Completed the [Quickstart](/documentation/get-started) to install and deploy NeMo Platform locally. +- Installed the Python SDK and any tutorial packages you need in your environment. +- A HuggingFace token with access to the private or gated model repository. +- A HuggingFace model with a compatible architecture. This tutorial uses `google/gemma-2-2b-it` as an example, but success depends on architectural compatibility. +- Sufficient GPU memory for the model and LoRA training job. -Verify that all required services are running and accessible before proceeding. You can check service health using the health endpoints documented in each service's API specification. +This tutorial uses the current NeMo Platform SDK resources: `client.files.filesets`, `client.models`, `client.customization.automodel.jobs`, and `client.inference.gateway`. + ### Known Issues @@ -42,266 +43,169 @@ Verify that all required services are running and accessible before proceeding. **Solution**: Use modern transformer architectures instead: -- ✅ **Recommended**: Llama models (3.1, 3.2, 3.3 series) -- ✅ **Recommended**: Nemotron models -- ✅ **Recommended**: Phi models -- ✅ **Alternative**: Gemma models (used in this tutorial) +- Llama models (3.1, 3.2, 3.3 series) +- Nemotron models +- Phi models +- Gemma models (used in this tutorial) For a complete list of tested models, see the [Model Catalog](/documentation/customizer-reference/models/model-catalog). ---- -## Download Model From HuggingFace Hub - -1. Authenticate to HuggingFace using `hf auth login`. -2. Download the model. - ---- +## Quick Start -## Create Model in Data Store +### 1. Initialize the SDK -Next, create a model repository in the NeMo Data Store and upload the downloaded model files. +The SDK needs your NeMo Platform server URL. By default, this tutorial uses `http://localhost:8080`. -### Create Namespace and Model Repository - -```bash -# Set environment variables - Update these to match your deployment -export NAMESPACE="my-org" -export MODEL_NAME="gemma-2-2b-it" -export MODEL_VERSION="$(date +%Y%m%d-%H%M%S)" # Unique version for this run -export NEMO_BASE_URL="http://nemo.test" -export DATASTORE_URL="http://data-store.test" -export REPO_ID="${NAMESPACE}/${MODEL_NAME}" -export DATASET_NAME="${MODEL_NAME}-training-data" - -# Create namespace -curl -X POST "${DATASTORE_URL}/v1/datastore/namespaces" \ --H 'Content-Type: application/json' \ --d '{"namespace": "'${NAMESPACE}'"}' - -# Create model repository in datastore -curl -X POST "${DATASTORE_URL}/v1/hf/api/repos/create" \ --H 'Content-Type: application/json' \ --d '{ -"organization": "'${NAMESPACE}'", -"name": "'${MODEL_NAME}'", -"type": "model" -}' +```sh +export NMP_BASE_URL= +export HF_TOKEN= ``` -### Upload Model Files to Data Store - -Upload the downloaded model files to the Data Store repository: - -## Create Model Entity in Entity Store +```python +import json +import os +import time +import uuid +from pathlib import Path + +from IPython.display import clear_output +from nemo_automodel_plugin.schema import ( + AutomodelJobInput, + BatchSpec, + DatasetSpec, + OptimizerSpec, + OutputRequest, + ParallelismSpec, + ScheduleSpec, + TrainingSpec, +) +from nemo_platform import ConflictError, NeMoPlatform +from nemo_platform.types.files import HuggingfaceStorageConfigParam +from nemo_platform.types.secrets import PlatformSecretResponse -After uploading the model files to the Data Store, create a model entity in the Entity Store to register the model with its metadata and specifications for use in customization jobs. -```bash -# Create model entity in Entity Store -curl -X POST "${NEMO_BASE_URL}/v1/models" \ --H 'Content-Type: application/json' \ --d '{ -"name": "'${MODEL_NAME}'", -"namespace": "'${NAMESPACE}'", -"description": "Private '${MODEL_NAME}' model imported for customization", -"artifact": { -"files_url": "hf://models/'${REPO_ID}'", -"backend_engine": "hugging_face", -"status": "upload_completed" -}, -"spec": { -"num_parameters": 200000000, -"context_size": 1024, -"is_chat": true, -"num_virtual_tokens": -1 -}, -"peft": { -"finetuning_type": "all_weights" -} -}' | jq . -``` +def max_wait_time_checker(seconds: int, job_name: str = ""): + start_time = time.time() -## Deploy the Base Model + def check(): + if time.time() - start_time > seconds: + raise TimeoutError(f"{job_name} took longer than {seconds} seconds") -Deploy the base model for inference with LoRA adapter support enabled, allowing it to load fine-tuned adapters from customization jobs. + return check -```bash -curl "${NEMO_BASE_URL}/v1/deployment/configs" \ --X POST \ --H 'Content-Type: application/json' \ ---data-binary '{ -"model": "'${MODEL_NAME}'", -"name": "'${MODEL_NAME}'-deployment-config", -"namespace": "'${NAMESPACE}'", -"nim_deployment": { -"additional_envs": { -"NIM_FT_MODEL": "", -"NIM_GUIDED_DECODING_BACKEND": "outlines", -"NIM_JSONL_LOGGING": "0", -"NIM_MODEL_NAME": "/model-store", -"NIM_PEFT_REFRESH_INTERVAL": "30", -"NIM_PEFT_SOURCE": "http://nemo-entity-store:8000", -"UVICORN_LOG_LEVEL": "DEBUG", -"VLLM_NVEXT_LOG_LEVEL": "DEBUG" -}, -"gpu": 1, -"image_name": "nvcr.io/nim/nvidia/llm-nim", -"image_tag": "1.13.1", -"disable_lora_support": false -} -}' | jq . - - -curl "${NEMO_BASE_URL}/v1/deployment/model-deployments" \ --X POST \ --H 'Content-Type: application/json' \ --d '{ -"name": "'${MODEL_NAME}'-deployment", -"namespace": "'${NAMESPACE}'", -"config": "'${NAMESPACE}'/'${MODEL_NAME}'-deployment-config" -}' | jq . +NMP_BASE_URL = os.environ.get("NMP_BASE_URL", "http://localhost:8080") +client = NeMoPlatform(base_url=NMP_BASE_URL, workspace="default") ``` -## Create Customization Target +### 2. Store the HuggingFace Token -Create a customization target that references the uploaded model in the Data Store. +Private and gated HuggingFace repositories require a token. Store it as a NeMo Platform secret and reference that secret from the HuggingFace fileset. -```bash - -# Create customization target -curl -X POST \ -"${NEMO_BASE_URL}/v1/customization/targets" \ --H 'accept: application/json' \ --H 'Content-Type: application/json' \ --d '{ -"name": "'${MODEL_NAME}'@v'${MODEL_VERSION}'", -"namespace": "'${NAMESPACE}'", -"description": "Customization target for '${MODEL_NAME}'", -"enabled": true, -"model_uri": "hf://'${NAMESPACE}'/'${MODEL_NAME}'", -"num_parameters": 200000000, -"precision": "bf16-mixed" -}' | jq . +```python +def create_or_get_secret(name: str, value: str | None, label: str) -> PlatformSecretResponse: + if not value: + raise ValueError(f"{label} is not set") + try: + secret = client.secrets.create( + name=name, + workspace="default", + value=value, + ) + print(f"Created secret: {name}") + return secret + except ConflictError: + print(f"Secret '{name}' already exists, continuing...") + return client.secrets.retrieve(name=name, workspace="default") + + +hf_secret = create_or_get_secret("hf-token", os.getenv("HF_TOKEN"), "HF_TOKEN") ``` -Wait for the model to be downloaded and ready: +### 3. Create a Model FileSet and Model Entity -```bash -# Check target status with comprehensive handling -while true; do -RESPONSE=$(curl -s -X GET \ -"${NEMO_BASE_URL}/v1/customization/targets/${NAMESPACE}/${MODEL_NAME}@v${MODEL_VERSION}" \ --H 'accept: application/json') - -STATUS=$(echo "$RESPONSE" | jq -r '.status') -echo "Target status: $STATUS" - -if [ "$STATUS" = "ready" ]; then -echo "Model is ready for customization!" -break -elif [ "$STATUS" = "failed" ] || [ "$STATUS" = "cancelled" ] || [ "$STATUS" = "unknown" ] || [ "$STATUS" = "delete_failed" ]; then -echo "Model download failed with status: $STATUS" -echo "Contact your administrator for assistance." -break -elif [ "$STATUS" = "created" ] || [ "$STATUS" = "pending" ] || [ "$STATUS" = "downloading" ]; then -echo "Model is still being prepared, waiting..." -elif [ "$STATUS" = "deleted" ] || [ "$STATUS" = "deleting" ]; then -echo "Model is being deleted (status: $STATUS)" -echo "This target cannot be used for customization." -break -else -echo "Unknown status: $STATUS" -fi - -sleep 30 -done +Create a HuggingFace-backed fileset for the private model, then register a Model Entity that points to that fileset. Model files are downloaded by the platform when training or deployment needs them. -``` +```python +HF_REPO_ID = "google/gemma-2-2b-it" +MODEL_NAME = "gemma-2-2b-it" +model_storage = HuggingfaceStorageConfigParam( + type="huggingface", + repo_id=HF_REPO_ID, + repo_type="model", + token_secret=hf_secret.name, +) -## Create Customization Configuration +try: + base_model_fs = client.files.filesets.create( + workspace="default", + name=MODEL_NAME, + description=f"Private HuggingFace model {HF_REPO_ID}", + storage=model_storage, + cache=True, + ) + print(f"Created model fileset: {base_model_fs.name}") +except ConflictError: + print(f"Model fileset '{MODEL_NAME}' already exists, refreshing HuggingFace settings...") + client.files.filesets.delete(workspace="default", name=MODEL_NAME) + base_model_fs = client.files.filesets.create( + workspace="default", + name=MODEL_NAME, + description=f"Private HuggingFace model {HF_REPO_ID}", + storage=model_storage, + cache=True, + ) -Create a configuration for LoRA fine-tuning: +try: + base_model = client.models.create( + workspace="default", + name=MODEL_NAME, + fileset=f"default/{MODEL_NAME}", + trust_remote_code=False, + ) + print(f"Created Model Entity: {base_model.name}") +except ConflictError: + print(f"Model Entity '{MODEL_NAME}' already exists, updating fileset if needed...") + base_model = client.models.update( + workspace="default", + name=MODEL_NAME, + fileset=f"default/{MODEL_NAME}", + trust_remote_code=False, + ) -```bash -# Create customization configuration -curl -X POST \ -"${NEMO_BASE_URL}/v1/customization/configs" \ --H 'accept: application/json' \ --H 'Content-Type: application/json' \ --d '{ -"name": "'${MODEL_NAME}'-lora-config@v'${MODEL_VERSION}'", -"namespace": "'${NAMESPACE}'", -"target": "'${NAMESPACE}'/'${MODEL_NAME}'@v'${MODEL_VERSION}'", -"description": "LoRA configuration for '${MODEL_NAME}'", -"training_options": [ -{ -"training_type": "sft", -"finetuning_type": "lora", -"num_gpus": 1, -"num_nodes": 1, -"tensor_parallel_size": 1, -"pipeline_parallel_size": 1, -"micro_batch_size": 1 -} -], -"training_precision": "bf16-mixed", -"max_seq_length": 1024, -"prompt_template": "{prompt} {completion}" -}' | jq . +time_check = max_wait_time_checker(600, "ModelSpec") +while not base_model.spec: + time_check() + time.sleep(10) + base_model = client.models.retrieve(workspace="default", name=MODEL_NAME) + +if base_model.spec: + base_model.spec.linear_layers = None + base_model = client.models.update( + workspace="default", + name=MODEL_NAME, + spec=base_model.spec, + verbose=True, + ) +print(base_model.model_dump_json(indent=2)) ``` ---- - -## Prepare Training and Validation Datasets - -Before starting the customization job, prepare both training and validation datasets. The validation dataset helps track training progress and reduce overfitting. +### 4. Prepare and Upload Training Data -Create datasets in JSONL format: +Create chat-format JSONL files and upload them to a fileset. The file service, not a dataset-specific SDK resource, is the current entry point for training data. ```python -import json -# Note: This assumes MODEL_NAME, NAMESPACE, and hf_api are already defined from previous code blocks +DATASET_NAME = f"{MODEL_NAME}-training-data" +DATASET_PATH = Path(DATASET_NAME).absolute() +DATASET_PATH.mkdir(parents=True, exist_ok=True) -# Example training data for Gemma 2 (instruction format) -# Gemma 2 uses standard chat message format training_data = [ { "messages": [ {"role": "user", "content": "Hello, how are you?"}, - { - "role": "assistant", - "content": "I'm doing well, thank you for asking! How can I assist you today?", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "What's the weather like?"}, - { - "role": "assistant", - "content": "I don't have access to current weather data, but I can help you find weather information or discuss weather patterns.", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "Tell me a joke"}, - { - "role": "assistant", - "content": "Why don't scientists trust atoms? Because they make up everything!", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "Can you help me with coding?"}, - { - "role": "assistant", - "content": "Absolutely! I'd be happy to help you with coding. What programming language or specific problem are you working on?", - }, + {"role": "assistant", "content": "I'm doing well. How can I help you today?"}, ] }, { @@ -309,380 +213,267 @@ training_data = [ {"role": "user", "content": "What is machine learning?"}, { "role": "assistant", - "content": "Machine learning is a subset of artificial intelligence that enables computers to learn and make decisions from data without being explicitly programmed.", + "content": "Machine learning is a way for computers to learn patterns from data.", }, ] }, { "messages": [ - {"role": "user", "content": "How do I write a function in Python?"}, - { - "role": "assistant", - "content": "To write a function in Python, use the 'def' keyword followed by the function name and parameters: def my_function(param): return result", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "What are the benefits of exercise?"}, - { - "role": "assistant", - "content": "Exercise offers numerous benefits including improved cardiovascular health, stronger muscles, better mental health, and increased energy levels.", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "Explain the solar system"}, - { - "role": "assistant", - "content": "The solar system consists of the Sun and all celestial objects that orbit it, including eight planets, moons, asteroids, and comets.", - }, + {"role": "user", "content": "Can you help me with Python?"}, + {"role": "assistant", "content": "Yes. Share the code or problem you want to work on."}, ] }, ] -# Expand training data by repeating examples to create a larger dataset -training_data = training_data * 10 # 80 total examples (8 unique * 10 repetitions) - -# Save training data to JSONL file -with open("training_data.jsonl", "w") as f: - for item in training_data: - f.write(json.dumps(item) + "\n") - -# Create dataset in datastore (similar to model creation) -dataset_name = f"{MODEL_NAME}-training-data" -dataset_repo_id = f"{NAMESPACE}/{dataset_name}" - -# Create dataset repository -hf_api.create_repo(dataset_repo_id, repo_type="dataset", exist_ok=True) - -# Upload training data -hf_api.upload_file( - path_or_fileobj="training_data.jsonl", - path_in_repo="training.jsonl", - repo_id=dataset_repo_id, - repo_type="dataset", - commit_message="Upload training data", -) - validation_data = [ - { - "messages": [ - {"role": "user", "content": "How do you stay motivated?"}, - { - "role": "assistant", - "content": "Staying motivated involves setting clear goals, celebrating small wins, and maintaining a positive mindset even when facing challenges.", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "What's the capital of France?"}, - { - "role": "assistant", - "content": "The capital of France is Paris, a beautiful city known for its art, culture, and iconic landmarks like the Eiffel Tower.", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "Explain quantum physics"}, - { - "role": "assistant", - "content": "Quantum physics is the branch of physics that studies matter and energy at the smallest scales, where particles behave in ways that seem strange compared to our everyday experience.", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "What is photosynthesis?"}, - { - "role": "assistant", - "content": "Photosynthesis is the process by which plants convert sunlight, carbon dioxide, and water into glucose and oxygen, providing energy for the plant.", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "How does the internet work?"}, - { - "role": "assistant", - "content": "The internet works through a global network of interconnected computers that communicate using standardized protocols to share information and resources.", - }, - ] - }, { "messages": [ {"role": "user", "content": "What is artificial intelligence?"}, { "role": "assistant", - "content": "Artificial intelligence is the simulation of human intelligence in machines that are programmed to think, learn, and solve problems like humans.", - }, - ] - }, - { - "messages": [ - {"role": "user", "content": "Describe the water cycle"}, - { - "role": "assistant", - "content": "The water cycle is the continuous movement of water through evaporation, condensation, precipitation, and collection that sustains life on Earth.", + "content": "Artificial intelligence is software that performs tasks associated with human intelligence.", }, ] }, { "messages": [ - {"role": "user", "content": "What are renewable energy sources?"}, + {"role": "user", "content": "Explain renewable energy."}, { "role": "assistant", - "content": "Renewable energy sources include solar, wind, hydroelectric, geothermal, and biomass energy that can be naturally replenished and don't deplete over time.", + "content": "Renewable energy comes from naturally replenished sources such as sunlight, wind, and water.", }, ] }, ] -# Expand validation data by repeating examples to create a larger dataset -validation_data = validation_data * 10 # 80 total examples (8 unique * 10 repetitions) +with open(DATASET_PATH / "training.jsonl", "w", encoding="utf-8") as f: + for item in training_data: + f.write(json.dumps(item) + "\n") -# Save validation data to JSONL file -with open("validation_data.jsonl", "w") as f: +with open(DATASET_PATH / "validation.jsonl", "w", encoding="utf-8") as f: for item in validation_data: f.write(json.dumps(item) + "\n") -# Upload validation data -hf_api.upload_file( - path_or_fileobj="validation_data.jsonl", - path_in_repo="validation.jsonl", - repo_id=dataset_repo_id, - repo_type="dataset", - commit_message="Upload validation data", -) - -# Create dataset entity in Entity Store -dataset = client.datasets.create( - name=dataset_name, namespace=NAMESPACE, files_url=f"hf://datasets/{dataset_repo_id}" +try: + client.files.filesets.create( + workspace="default", + name=DATASET_NAME, + description="Private HuggingFace model LoRA training data", + cache=True, + ) + print(f"Created dataset fileset: {DATASET_NAME}") +except ConflictError: + print(f"Dataset fileset '{DATASET_NAME}' already exists, continuing...") + +client.files.fsspec.put( + lpath=DATASET_PATH, + rpath=f"default/{DATASET_NAME}/", + recursive=True, ) -print(f"Created dataset entity: {dataset.namespace}/{dataset.name}") +print(client.files.list(fileset=DATASET_NAME, workspace="default")) ``` ---- - -## Start Customization Job +### 5. Start a LoRA Customization Job -Start the LoRA fine-tuning job. The job will create an output artifact with the name specified in `output`, which you'll use later to access your fine-tuned model for inference. +Submit a LoRA job to the Automodel backend. Reference the model and dataset filesets using `workspace/name`. -```bash -# Create job and capture job ID -RESPONSE=$(curl -s -X POST \ -"${NEMO_BASE_URL}/v1/customization/jobs" \ --H 'accept: application/json' \ --H 'Content-Type: application/json' \ --d '{ -"name": "'${MODEL_NAME}'-lora-job", -"config": "'${NAMESPACE}'/'${MODEL_NAME}'-lora-config@v'${MODEL_VERSION}'", -"dataset": "'${NAMESPACE}'/'${DATASET_NAME}'", -"output": {"name": "'${NAMESPACE}'/'${MODEL_NAME}'-lora@v'${MODEL_VERSION}'"}, -"description": "LoRA fine-tuning job for '${MODEL_NAME}'", -"training": { -"type": "sft", -"peft": { -"type": "lora", -"rank": 16, -"alpha": 32, -"dropout": 0.01 -}, -"epochs": 3, -"batch_size": 8, -"learning_rate": 5e-5 -} -}') - -JOB_ID=$(echo "$RESPONSE" | jq -r '.id') -OUTPUT_NAME=$(echo "$RESPONSE" | jq -r '.spec.output.name') -echo "Started job with ID: $JOB_ID" -echo "Output name: $OUTPUT_NAME" -``` +```python +job_suffix = uuid.uuid4().hex[:4] +JOB_NAME = f"{MODEL_NAME}-lora-job-{job_suffix}" +OUTPUT_NAME = f"{MODEL_NAME}-lora-{job_suffix}" + +spec = AutomodelJobInput( + model=f"default/{base_model.name}", + dataset=DatasetSpec( + training=f"default/{DATASET_NAME}", + validation=f"default/{DATASET_NAME}", + ), + training=TrainingSpec( + training_type="sft", + finetuning_type="lora", + max_seq_length=2048, + ), + schedule=ScheduleSpec(epochs=1), + batch=BatchSpec(global_batch_size=64, micro_batch_size=1), + optimizer=OptimizerSpec(learning_rate=5e-5), + parallelism=ParallelismSpec( + num_gpus_per_node=1, + num_nodes=1, + tensor_parallel_size=1, + pipeline_parallel_size=1, + context_parallel_size=1, + expert_parallel_size=1, + ), + output=OutputRequest(name=OUTPUT_NAME), +) -Copy the following values from the response: +job = client.customization.automodel.jobs.create( + spec=spec, + workspace="default", + name=JOB_NAME, +) -- `id` (Job ID) -- `spec.output.name` +print(f"Submitted job: {job.job.name}") +print(f"Output adapter: {OUTPUT_NAME}") +``` -We'll need them later to monitor the job's status and access the fine-tuned model. +### 6. Track Training Progress -Check job progress: +Poll job status until training reaches a terminal state. -```bash -# Monitor job status with comprehensive handling -while true; do -RESPONSE=$(curl -s -X GET \ -"${NEMO_BASE_URL}/v1/customization/jobs/${JOB_ID}" \ --H 'accept: application/json') - -STATUS=$(echo "$RESPONSE" | jq -r '.status') -echo "Job status: $STATUS" - -if [ "$STATUS" = "completed" ]; then -echo "Training completed successfully!" -break -elif [ "$STATUS" = "failed" ] || [ "$STATUS" = "cancelled" ]; then -echo "Training finished with status: $STATUS" -if [ "$STATUS" = "failed" ]; then -echo "Check the job logs for error details." -fi -break -elif [ "$STATUS" = "created" ] || [ "$STATUS" = "pending" ]; then -echo "Job is queued and waiting to start..." -elif [ "$STATUS" = "running" ]; then -echo "Training is in progress..." -# Optionally show progress if available -PROGRESS=$(echo "$RESPONSE" | jq -r '.status_details.percentage_done // "N/A"') -if [ "$PROGRESS" != "N/A" ] && [ "$PROGRESS" != "null" ]; then -echo "Progress: ${PROGRESS}%" -fi -elif [ "$STATUS" = "cancelling" ]; then -echo "Job is being cancelled..." -elif [ "$STATUS" = "ready" ] || [ "$STATUS" = "unknown" ]; then -echo "Job finished with status: $STATUS" -break -else -echo "Unknown status: $STATUS" -fi - -sleep 60 # Wait 1 minute before checking again -done +```python +time_check = max_wait_time_checker(3600, "Customization Job") + +while True: + time_check() + status = client.jobs.get_status(name=job.job.name, workspace="default") + clear_output(wait=True) + print(f"Job status: {status.status}") + + step = max_steps = None + training_phase = None + for job_step in status.steps or []: + if job_step.name == "training": + for task in job_step.tasks or []: + details = task.status_details if isinstance(task.status_details, dict) else {} + step = details.get("step") + max_steps = details.get("max_steps") + training_phase = details.get("phase") + break + break + + if isinstance(step, (int, float)) and isinstance(max_steps, (int, float)) and max_steps: + print(f"Training: step {step}/{max_steps} ({100 * step / max_steps:.1f}%)") + if isinstance(training_phase, str): + print(f"Phase: {training_phase}") + + if status.status in ("completed", "failed", "cancelled", "error"): + print(f"\nJob finished: {status.status}") + break + + time.sleep(10) + +if status.status != "completed": + raise RuntimeError(f"Training job finished with status: {status.status}") ``` ---- +### 7. Deploy the Base Model With LoRA Enabled -## Test the Deployed Model +Create a deployment for the base model with LoRA support enabled. The LoRA adapter from training is served by the same deployment. -After the customization job has been completed, you can use the `output.name` to access the fine-tuned model and evaluate its performance. The base model NIM deployment you created earlier will automatically load the LoRA adapter when you specify the LoRA model ID in your inference requests. +```python +deploy_suffix = uuid.uuid4().hex[:4] +DEPLOYMENT_CONFIG_NAME = f"private-hf-lora-cfg-{deploy_suffix}" +DEPLOYMENT_NAME = f"private-hf-lora-{deploy_suffix}" + +deployment_config = client.inference.deployment_configs.create( + workspace="default", + name=DEPLOYMENT_CONFIG_NAME, + engine="vllm", + model_spec={ + "model_namespace": "default", + "model_name": MODEL_NAME, + "lora_enabled": True, + }, + executor_config={ + "gpu": 1, + "image_name": "vllm/vllm-openai", + "image_tag": "v0.22.1", + "additional_args": ["--max-lora-rank", "32"], + }, +) - +deployment = client.inference.deployments.create( + workspace="default", + name=DEPLOYMENT_NAME, + config=deployment_config.name, +) -The inference endpoints use the `inference_base_url` configured during client initialization (typically the NIM proxy URL). The base model deployment must be running before you can test inference with LoRA adapters. +print(f"Deployment name: {deployment.name}") +print(f"Deployment status: {deployment.status}") +``` - - +```python +time_check = max_wait_time_checker(1800, "Deployment") +while True: + time_check() + time.sleep(15) + deployment_status = client.inference.deployments.retrieve( + name=deployment.name, + workspace="default", + ) + clear_output(wait=True) + print(f"Deployment: {deployment.name}") + print(f"Status: {deployment_status.status}") + + if deployment_status.status in ("RUNNING", "READY"): + if not client.models.wait_for_gateway(deployment.name, workspace="default", timeout=60): + raise RuntimeError("Inference gateway did not become ready") + break + if deployment_status.status in ("FAILED", "ERROR", "TERMINATED", "LOST"): + raise RuntimeError(f"Deployment failed with status: {deployment_status.status}") +``` -If you included a WandB API key, you can view your training results at [wandb.ai](https://wandb.ai/home) under the `nvidia-nemo-customizer` project. +### 8. Test the Deployed Model + +Call the inference gateway through the SDK. Use the base model name to test the original model and the output adapter name to test the LoRA-adapted model. - ```python -base_model_id = f"{NAMESPACE}/{MODEL_NAME}" - -# Option 1: If you still have the job object from creation -# lora_model_id = OUTPUT_NAME +messages = [ + {"role": "user", "content": "Can you summarize what LoRA fine-tuning does?"} +] -# Option 2: Construct from job parameters (use this if running in a new session) -lora_model_id = f"{NAMESPACE}/{MODEL_NAME}-lora@v{MODEL_VERSION}" -# First, check if the models are available for inference -# Note: The base model deployment must be running and registered with the NIM proxy -try: - # List models from the entity store (shows all registered models) - models_response = client.models.list() - available_models = models_response.data - - print("Registered models:") - for model in available_models: - print(f" - {model.id}") - - # Test base model - print(f"\nTesting base model: {base_model_id}") - base_response = client.chat.completions.create( - model=base_model_id, - messages=[ - { - "role": "user", - "content": "Hello, can you help me?" - } - ], - max_tokens=100, - temperature=0.7 +def chat(model_id: str): + return client.inference.gateway.provider.post( + "v1/chat/completions", + name=deployment.name, + workspace="default", + body={ + "model": model_id, + "messages": messages, + "temperature": 0.2, + "max_tokens": 128, + }, ) - print("Base model response:") - print(base_response.choices[0].message.content) - # Test LoRA-adapted model (if available) - print(f"\nTesting LoRA-adapted model: {lora_model_id}") - lora_response = client.chat.completions.create( - model=lora_model_id, - messages=[ - { - "role": "user", - "content": "Hello, can you help me?" - } - ], - max_tokens=100, - temperature=0.7 - ) - - print("LoRA-adapted model response:") - print(lora_response.choices[0].message.content) +base_response = chat(f"default/{MODEL_NAME}") +lora_response = chat(f"default--{OUTPUT_NAME}") -except Exception as e: - print(f"Error testing model: {e}") +print("Base model response:") +print(base_response["choices"][0]["message"]["content"]) +print("\nLoRA-adapted model response:") +print(lora_response["choices"][0]["message"]["content"]) ``` + ```bash -# Set the NIM proxy URL -export NIM_PROXY_URL="http://nim.test" - -# Option 1: If you captured OUTPUT_MODEL from job creation (from earlier in tutorial) -# export OUTPUT_MODEL="" - -# Option 2: Construct from environment variables (use this if running in a new session) -export LORA_MODEL_ID="${NAMESPACE}/${MODEL_NAME}-lora@v${MODEL_VERSION}" - -# Test model availability -curl -X GET "${NIM_PROXY_URL}/models" | jq - -# Test inference against the base model -curl -X POST "${NIM_PROXY_URL}/v1/chat/completions" \ --H 'Content-Type: application/json' \ --d '{ -"model": "'${NAMESPACE}'/'${MODEL_NAME}'", -"messages": [ -{"role": "user", "content": "Hello! How are you?"}, -{"role": "assistant", "content": "Hi! I am quite well, how can I help you today?"}, -{"role": "user", "content": "Can you write me a song?"} -], -"top_p": 1, -"n": 1, -"max_tokens": 50, -"frequency_penalty": 1.0 -}' - -# Test inference against a LoRA-adapted model (after customization completes) -curl -X POST "${NIM_PROXY_URL}/v1/chat/completions" \ --H 'Content-Type: application/json' \ --d '{ -"model": "'${LORA_MODEL_ID}'", -"messages": [ -{"role": "user", "content": "Hello! How are you?"}, -{"role": "assistant", "content": "Hi! I am quite well, how can I help you today?"}, -{"role": "user", "content": "Can you write me a song?"} -], -"top_p": 1, -"n": 1, -"max_tokens": 50, -"frequency_penalty": 1.0 -}' +export OUTPUT_NAME="" + +curl -s "${NMP_BASE_URL}/apis/inference-gateway/v2/workspaces/default/openai/-/v1/chat/completions" \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "'${OUTPUT_NAME}'", + "messages": [ + {"role": "user", "content": "Can you summarize what LoRA fine-tuning does?"} + ], + "temperature": 0.2, + "max_tokens": 128 + }' | jq ``` + ## Next Steps -Learn how to [check customization job metrics](/documentation/customizer-reference/tutorials/metrics) to monitor the training progress and performance of your fine-tuned model. +Learn how to [check customization job metrics](/documentation/customizer-reference/tutorials/metrics) to monitor training progress and performance for your fine-tuned model. diff --git a/docs/customizer/tutorials/index.mdx b/docs/customizer/tutorials/index.mdx index 6214b79930..f734e48412 100644 --- a/docs/customizer/tutorials/index.mdx +++ b/docs/customizer/tutorials/index.mdx @@ -53,7 +53,7 @@ Learn how to perform supervised fine-tuning with LoRA adapters using custom data -Learn how to perform supervised fine-tuning using custom data by modifying the all training parameters. +Learn how to perform supervised fine-tuning using custom data by modifying all training parameters. nemo-customizer diff --git a/docs/customizer/tutorials/optimize-throughput.ipynb b/docs/customizer/tutorials/optimize-throughput.ipynb index ed69b9ce27..a4b6171f91 100644 --- a/docs/customizer/tutorials/optimize-throughput.ipynb +++ b/docs/customizer/tutorials/optimize-throughput.ipynb @@ -98,7 +98,6 @@ }, "outputs": [], "source": [ - "%%bash\n", "if command -v uv >/dev/null 2>&1 && [ -n \"$VIRTUAL_ENV\" ]; then\n", " uv pip install datasets pandas matplotlib nvidia-ml-py\n", "else\n", diff --git a/docs/customizer/tutorials/optimize-throughput.mdx b/docs/customizer/tutorials/optimize-throughput.mdx index 20dc00fdf5..fa87e0fea4 100644 --- a/docs/customizer/tutorials/optimize-throughput.mdx +++ b/docs/customizer/tutorials/optimize-throughput.mdx @@ -61,7 +61,6 @@ Required packages: - `nvidia-ml-py` - Collect GPU VRAM and compute utilization metrics during training ```sh -%%bash if command -v uv >/dev/null 2>&1 && [ -n "$VIRTUAL_ENV" ]; then uv pip install datasets pandas matplotlib nvidia-ml-py else diff --git a/docs/customizer/tutorials/sft-customization-job.ipynb b/docs/customizer/tutorials/sft-customization-job.ipynb index 0dc0515ad2..4a85bc9a81 100644 --- a/docs/customizer/tutorials/sft-customization-job.ipynb +++ b/docs/customizer/tutorials/sft-customization-job.ipynb @@ -772,17 +772,15 @@ "context = \"The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit.\"\n", "question = \"Who was the first person to walk on the Moon?\"\n", "\n", - "messages = [\n", - " {\"role\": \"user\", \"content\": f\"Based on the following context, answer the question.\\n\\nContext: {context}\\n\\nQuestion: {question}\"}\n", - "]\n", + "prompt = f\"Context: {context} Question: {question} Answer:\"\n", "\n", "response = client.inference.gateway.provider.post(\n", - " \"v1/chat/completions\",\n", + " \"v1/completions\",\n", " name=deployment.name,\n", " workspace=\"default\",\n", " body={\n", " \"model\": f\"default/{OUTPUT_NAME}\",\n", - " \"messages\": messages,\n", + " \"prompt\": prompt,\n", " \"temperature\": 0,\n", " \"max_tokens\": 128\n", " }\n", @@ -793,7 +791,7 @@ "print(\"=\" * 60)\n", "print(f\"Question: {question}\")\n", "print(f\"Expected: Neil Armstrong\")\n", - "print(f\"Model output: {response['choices'][0]['message']['content']}\")" + "print(f\"Model output: {response['choices'][0]['text']}\")" ], "execution_count": null, "outputs": [] diff --git a/docs/customizer/tutorials/sft-customization-job.mdx b/docs/customizer/tutorials/sft-customization-job.mdx index ea470754c7..2aac59c0ff 100644 --- a/docs/customizer/tutorials/sft-customization-job.mdx +++ b/docs/customizer/tutorials/sft-customization-job.mdx @@ -241,7 +241,7 @@ Get your credentials to access base models: #### Quick Setup Example -In this tutorial we are going to work with [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/tree/main) model from HuggingFace. Ensure that you have sufficient permissions to download the model. If you cannot access the files on the [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/tree/main) Hugging Face page, request access +In this tutorial we are going to work with [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/tree/main) model from HuggingFace. Ensure that you have sufficient permissions to download the model. If you cannot access the files on the [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/tree/main) Hugging Face page, request access. **HuggingFace Authentication:** - For gated models (Llama, Gemma), you must provide a HuggingFace token via the `token_secret` parameter @@ -606,17 +606,15 @@ After training, evaluate whether your model meets your requirements: context = "The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit." question = "Who was the first person to walk on the Moon?" -messages = [ - {"role": "user", "content": f"Based on the following context, answer the question.\n\nContext: {context}\n\nQuestion: {question}"} -] +prompt = f"Context: {context} Question: {question} Answer:" response = client.inference.gateway.provider.post( - "v1/chat/completions", + "v1/completions", name=deployment.name, workspace="default", body={ "model": f"default/{OUTPUT_NAME}", - "messages": messages, + "prompt": prompt, "temperature": 0, "max_tokens": 128 } @@ -627,7 +625,7 @@ print("MODEL EVALUATION") print("=" * 60) print(f"Question: {question}") print(f"Expected: Neil Armstrong") -print(f"Model output: {response['choices'][0]['message']['content']}") +print(f"Model output: {response['choices'][0]['text']}") ``` #### Evaluation Best Practices diff --git a/docs/customizer/tutorials/understand-configurations-and-models.mdx b/docs/customizer/tutorials/understand-configurations-and-models.mdx index 5eb1b09d5b..e9b9aa725d 100644 --- a/docs/customizer/tutorials/understand-configurations-and-models.mdx +++ b/docs/customizer/tutorials/understand-configurations-and-models.mdx @@ -13,7 +13,9 @@ Understanding these basics will help you navigate the fine-tuning process more e The time to complete this tutorial is approximately 15 minutes. - This tutorial focuses on understanding and discovery—no actual training jobs are created. + +This tutorial focuses on understanding and discovery—no actual training jobs are created. + ## Prerequisites @@ -60,7 +62,7 @@ flowchart LR A[1. Create FileSet
with model files] --> B[2. Create Model Entity
pointing to FileSet] B --> C[3. Create Customization Job
referencing Model Entity] C --> D{Training Type?} - D -->{{ LoRA }} E[Adapter created and
attached to Model Entity] + D -->|LoRA| E[Adapter created and
attached to Model Entity] D -->|Full SFT| F[New Model Entity
with customized weights] E --> G[Auto-deploy to NIM
if enabled] @@ -184,10 +186,10 @@ for adapter in model.adapters or []: Adapters are enabled by default, but you can disable an adapter to remove it from inference without deleting it. When you set `enabled=False`, the sidecar running alongside the NIM automatically removes the adapter's files on its next reconciliation pass (every few seconds). Re-enabling the adapter causes the sidecar to re-download and serve it again. ```python -client.models.update_adapter( +client.models.adapters.update( + adapter="my-custom-lora", model_name="llama-3-2-1b", workspace="default", - adapter_name="my-custom-lora", enabled=False, ) ``` @@ -195,10 +197,10 @@ client.models.update_adapter( To re-enable: ```python -client.models.update_adapter( +client.models.adapters.update( + adapter="my-custom-lora", model_name="llama-3-2-1b", workspace="default", - adapter_name="my-custom-lora", enabled=True, ) ``` @@ -208,7 +210,7 @@ client.models.update_adapter( You can also create adapters manually (e.g., from externally trained weights): ```python -model = client.models.create_adapter( +model = client.models.adapters.create( model_name="llama-3-2-1b", workspace="default", name="my-custom-lora", @@ -314,15 +316,15 @@ If this constraint isn't met, your training job will fail with a validation erro flowchart TD A[What's your goal?] --> B{Need maximum
performance?} - B -->{{ Yes }} C{Have 4+ GPUs?} - B -->{{ No }} D[Choose LoRA] + B -->|Yes| C{Have 4+ GPUs?} + B -->|No| D[Choose LoRA] - C -->{{ Yes }} E[Choose Full SFT] - C -->{{ No }} D + C -->|Yes| E[Choose Full SFT] + C -->|No| D D --> F{Multiple
use cases?} - F -->{{ Yes }} G[Train multiple
LoRA adapters] - F -->{{ No }} H[Single LoRA
adapter] + F -->|Yes| G[Train multiple
LoRA adapters] + F -->|No| H[Single LoRA
adapter] E --> I[New Model Entity
with full weights] G --> J[Multiple adapters
on same Model Entity] @@ -462,7 +464,7 @@ job = client.customization.automodel.jobs.create( ) # Add an adapter to a model -client.models.create_adapter( +client.models.adapters.create( model_name="llama-3-2-1b", workspace="default", name="my-adapter", diff --git a/docs/fern/components/notebooks/embedding-customization-job.json b/docs/fern/components/notebooks/embedding-customization-job.json index 0f4e98cd95..fda1255a6e 100644 --- a/docs/fern/components/notebooks/embedding-customization-job.json +++ b/docs/fern/components/notebooks/embedding-customization-job.json @@ -91,8 +91,8 @@ }, { "type": "markdown", - "source": "### 6. Secrets Setup\n\nConfigure authentication for accessing base models:\n\n- **NGC models** (`ngc://` URIs): Requires NGC API key\n- **HuggingFace models** (`hf://` URIs): Requires HF token for gated/private models\n\nGet your credentials:\n- [NGC API Key](https://ngc.nvidia.com/) (Setup → Generate API Key)\n- [HuggingFace Token](https://huggingface.co/settings/tokens) (Create token with Read access)\n\n---\n\n#### Quick Setup Example\n\nThis tutorial fine-tunes [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2), NVIDIA embedding model optimized for question-answering and retrieval tasks.", - "source_html": "

6. Secrets Setup

\n

Configure authentication for accessing base models:

\n
    \n
  • NGC models (ngc:// URIs): Requires NGC API key
  • \n
  • HuggingFace models (hf:// URIs): Requires HF token for gated/private models
  • \n
\n

Get your credentials:

\n\n
\n

Quick Setup Example

\n

This tutorial fine-tunes nvidia/llama-3.2-nv-embedqa-1b-v2, NVIDIA embedding model optimized for question-answering and retrieval tasks.

\n" + "source": "### 6. Secrets Setup\n\nConfigure authentication for accessing base models:\n\n- **NGC models** (`ngc://` URIs): Requires NGC API key\n- **HuggingFace models** (`hf://` URIs): Requires HF token for gated/private models\n\nGet your credentials:\n- [NGC API Key](https://ngc.nvidia.com/) (Setup → Generate API Key)\n- [HuggingFace Token](https://huggingface.co/settings/tokens) (Create token with Read access)\n\n---\n\n#### Quick Setup Example\n\nThis tutorial fine-tunes [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2), an NVIDIA embedding model optimized for question-answering and retrieval tasks.", + "source_html": "

6. Secrets Setup

\n

Configure authentication for accessing base models:

\n
    \n
  • NGC models (ngc:// URIs): Requires NGC API key
  • \n
  • HuggingFace models (hf:// URIs): Requires HF token for gated/private models
  • \n
\n

Get your credentials:

\n\n
\n

Quick Setup Example

\n

This tutorial fine-tunes nvidia/llama-nemotron-embed-1b-v2, an NVIDIA embedding model optimized for question-answering and retrieval tasks.

\n" }, { "type": "code", @@ -102,14 +102,14 @@ }, { "type": "markdown", - "source": "### 7. Create Base Model FileSet and Model Entity\n\nCreate a fileset pointing to the [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.", - "source_html": "

7. Create Base Model FileSet and Model Entity

\n

Create a fileset pointing to the nvidia/llama-3.2-nv-embedqa-1b-v2 embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.

\n" + "source": "### 7. Create Base Model FileSet and Model Entity\n\nCreate a fileset pointing to the [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.", + "source_html": "

7. Create Base Model FileSet and Model Entity

\n

Create a fileset pointing to the nvidia/llama-nemotron-embed-1b-v2 embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.

\n" }, { "type": "code", - "source": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = \"nvidia/llama-nemotron-embed-1b-v2\"\nMODEL_NAME = \"nv-nemotron-embed-1b-base\"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace=\"default\",\n name=MODEL_NAME,\n description=\"NVIDIA Llama 3.2 NV EmbedQA 1B v2 embedding model\",\n storage=HuggingfaceStorageConfigParam(\n type=\"huggingface\",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type=\"model\",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f\"Base model fileset already exists. Skipping creation.\")\n base_model_fs = client.files.filesets.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n print(f\"Created Model Entity: {MODEL_NAME}\")\nexcept ConflictError:\n print(f\"Base model already exists. Updating fileset if different.\")\n base_model = client.models.update(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n\nprint(f\"\\nBase model fileset: fileset://default/{base_model.name}\")\nprint(\"\\nBase model files:\")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace=\"default\").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint(\"\\nWaiting for ModelSpec to be populated...\")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f\"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds\")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\nprint(f\"ModelSpec populated: {base_model.spec}\")", + "source": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = \"nvidia/llama-nemotron-embed-1b-v2\"\nMODEL_NAME = \"nv-nemotron-embed-1b-base\"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace=\"default\",\n name=MODEL_NAME,\n description=\"NVIDIA Llama Nemotron Embed 1B v2 embedding model\",\n storage=HuggingfaceStorageConfigParam(\n type=\"huggingface\",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type=\"model\",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f\"Base model fileset already exists. Skipping creation.\")\n base_model_fs = client.files.filesets.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n print(f\"Created Model Entity: {MODEL_NAME}\")\nexcept ConflictError:\n print(f\"Base model already exists. Updating fileset if different.\")\n base_model = client.models.update(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n\nprint(f\"\\nBase model fileset: fileset://default/{base_model.name}\")\nprint(\"\\nBase model files:\")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace=\"default\").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint(\"\\nWaiting for ModelSpec to be populated...\")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f\"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds\")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\nprint(f\"ModelSpec populated: {base_model.spec}\")", "language": "python", - "source_html": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = "nvidia/llama-nemotron-embed-1b-v2"\nMODEL_NAME = "nv-nemotron-embed-1b-base"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace="default",\n name=MODEL_NAME,\n description="NVIDIA Llama 3.2 NV EmbedQA 1B v2 embedding model",\n storage=HuggingfaceStorageConfigParam(\n type="huggingface",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type="model",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f"Base model fileset already exists. Skipping creation.")\n base_model_fs = client.files.filesets.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n print(f"Created Model Entity: {MODEL_NAME}")\nexcept ConflictError:\n print(f"Base model already exists. Updating fileset if different.")\n base_model = client.models.update(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n\nprint(f"\\nBase model fileset: fileset://default/{base_model.name}")\nprint("\\nBase model files:")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace="default").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint("\\nWaiting for ModelSpec to be populated...")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\nprint(f"ModelSpec populated: {base_model.spec}")\n" + "source_html": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = "nvidia/llama-nemotron-embed-1b-v2"\nMODEL_NAME = "nv-nemotron-embed-1b-base"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace="default",\n name=MODEL_NAME,\n description="NVIDIA Llama Nemotron Embed 1B v2 embedding model",\n storage=HuggingfaceStorageConfigParam(\n type="huggingface",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type="model",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f"Base model fileset already exists. Skipping creation.")\n base_model_fs = client.files.filesets.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n print(f"Created Model Entity: {MODEL_NAME}")\nexcept ConflictError:\n print(f"Base model already exists. Updating fileset if different.")\n base_model = client.models.update(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n\nprint(f"\\nBase model fileset: fileset://default/{base_model.name}")\nprint("\\nBase model files:")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace="default").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint("\\nWaiting for ModelSpec to be populated...")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\nprint(f"ModelSpec populated: {base_model.spec}")\n" }, { "type": "markdown", diff --git a/docs/fern/components/notebooks/embedding-customization-job.ts b/docs/fern/components/notebooks/embedding-customization-job.ts index 5b5a3aab41..0efe18943f 100644 --- a/docs/fern/components/notebooks/embedding-customization-job.ts +++ b/docs/fern/components/notebooks/embedding-customization-job.ts @@ -96,8 +96,8 @@ export default { cells: [ }, { "type": "markdown", - "source": "### 6. Secrets Setup\n\nConfigure authentication for accessing base models:\n\n- **NGC models** (`ngc://` URIs): Requires NGC API key\n- **HuggingFace models** (`hf://` URIs): Requires HF token for gated/private models\n\nGet your credentials:\n- [NGC API Key](https://ngc.nvidia.com/) (Setup → Generate API Key)\n- [HuggingFace Token](https://huggingface.co/settings/tokens) (Create token with Read access)\n\n---\n\n#### Quick Setup Example\n\nThis tutorial fine-tunes [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2), NVIDIA embedding model optimized for question-answering and retrieval tasks.", - "source_html": "

6. Secrets Setup

\n

Configure authentication for accessing base models:

\n
    \n
  • NGC models (ngc:// URIs): Requires NGC API key
  • \n
  • HuggingFace models (hf:// URIs): Requires HF token for gated/private models
  • \n
\n

Get your credentials:

\n\n
\n

Quick Setup Example

\n

This tutorial fine-tunes nvidia/llama-3.2-nv-embedqa-1b-v2, NVIDIA embedding model optimized for question-answering and retrieval tasks.

\n" + "source": "### 6. Secrets Setup\n\nConfigure authentication for accessing base models:\n\n- **NGC models** (`ngc://` URIs): Requires NGC API key\n- **HuggingFace models** (`hf://` URIs): Requires HF token for gated/private models\n\nGet your credentials:\n- [NGC API Key](https://ngc.nvidia.com/) (Setup → Generate API Key)\n- [HuggingFace Token](https://huggingface.co/settings/tokens) (Create token with Read access)\n\n---\n\n#### Quick Setup Example\n\nThis tutorial fine-tunes [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2), an NVIDIA embedding model optimized for question-answering and retrieval tasks.", + "source_html": "

6. Secrets Setup

\n

Configure authentication for accessing base models:

\n
    \n
  • NGC models (ngc:// URIs): Requires NGC API key
  • \n
  • HuggingFace models (hf:// URIs): Requires HF token for gated/private models
  • \n
\n

Get your credentials:

\n\n
\n

Quick Setup Example

\n

This tutorial fine-tunes nvidia/llama-nemotron-embed-1b-v2, an NVIDIA embedding model optimized for question-answering and retrieval tasks.

\n" }, { "type": "code", @@ -107,14 +107,14 @@ export default { cells: [ }, { "type": "markdown", - "source": "### 7. Create Base Model FileSet and Model Entity\n\nCreate a fileset pointing to the [nvidia/llama-3.2-nv-embedqa-1b-v2](https://huggingface.co/nvidia/llama-3.2-nv-embedqa-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.", - "source_html": "

7. Create Base Model FileSet and Model Entity

\n

Create a fileset pointing to the nvidia/llama-3.2-nv-embedqa-1b-v2 embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.

\n" + "source": "### 7. Create Base Model FileSet and Model Entity\n\nCreate a fileset pointing to the [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2) embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.", + "source_html": "

7. Create Base Model FileSet and Model Entity

\n

Create a fileset pointing to the nvidia/llama-nemotron-embed-1b-v2 embedding model from HuggingFace, then create a Model Entity that references this fileset. Model downloading will take place at training time.

\n" }, { "type": "code", - "source": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = \"nvidia/llama-nemotron-embed-1b-v2\"\nMODEL_NAME = \"nv-nemotron-embed-1b-base\"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace=\"default\",\n name=MODEL_NAME,\n description=\"NVIDIA Llama 3.2 NV EmbedQA 1B v2 embedding model\",\n storage=HuggingfaceStorageConfigParam(\n type=\"huggingface\",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type=\"model\",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f\"Base model fileset already exists. Skipping creation.\")\n base_model_fs = client.files.filesets.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n print(f\"Created Model Entity: {MODEL_NAME}\")\nexcept ConflictError:\n print(f\"Base model already exists. Updating fileset if different.\")\n base_model = client.models.update(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n\nprint(f\"\\nBase model fileset: fileset://default/{base_model.name}\")\nprint(\"\\nBase model files:\")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace=\"default\").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint(\"\\nWaiting for ModelSpec to be populated...\")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f\"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds\")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\nprint(f\"ModelSpec populated: {base_model.spec}\")", + "source": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = \"nvidia/llama-nemotron-embed-1b-v2\"\nMODEL_NAME = \"nv-nemotron-embed-1b-base\"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace=\"default\",\n name=MODEL_NAME,\n description=\"NVIDIA Llama Nemotron Embed 1B v2 embedding model\",\n storage=HuggingfaceStorageConfigParam(\n type=\"huggingface\",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type=\"model\",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f\"Base model fileset already exists. Skipping creation.\")\n base_model_fs = client.files.filesets.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n print(f\"Created Model Entity: {MODEL_NAME}\")\nexcept ConflictError:\n print(f\"Base model already exists. Updating fileset if different.\")\n base_model = client.models.update(\n workspace=\"default\",\n name=MODEL_NAME,\n fileset=f\"default/{MODEL_NAME}\",\n trust_remote_code=True,\n )\n\nprint(f\"\\nBase model fileset: fileset://default/{base_model.name}\")\nprint(\"\\nBase model files:\")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace=\"default\").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint(\"\\nWaiting for ModelSpec to be populated...\")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f\"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds\")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace=\"default\",\n name=MODEL_NAME,\n )\n\nprint(f\"ModelSpec populated: {base_model.spec}\")", "language": "python", - "source_html": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = "nvidia/llama-nemotron-embed-1b-v2"\nMODEL_NAME = "nv-nemotron-embed-1b-base"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace="default",\n name=MODEL_NAME,\n description="NVIDIA Llama 3.2 NV EmbedQA 1B v2 embedding model",\n storage=HuggingfaceStorageConfigParam(\n type="huggingface",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type="model",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f"Base model fileset already exists. Skipping creation.")\n base_model_fs = client.files.filesets.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n print(f"Created Model Entity: {MODEL_NAME}")\nexcept ConflictError:\n print(f"Base model already exists. Updating fileset if different.")\n base_model = client.models.update(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n\nprint(f"\\nBase model fileset: fileset://default/{base_model.name}")\nprint("\\nBase model files:")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace="default").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint("\\nWaiting for ModelSpec to be populated...")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\nprint(f"ModelSpec populated: {base_model.spec}")\n" + "source_html": "import time\nfrom nemo_platform.types.files import HuggingfaceStorageConfigParam\n\nHF_REPO_ID = "nvidia/llama-nemotron-embed-1b-v2"\nMODEL_NAME = "nv-nemotron-embed-1b-base"\n\n# Ensure you have a HuggingFace token secret created\ntry:\n base_model_fs = client.files.filesets.create(\n workspace="default",\n name=MODEL_NAME,\n description="NVIDIA Llama Nemotron Embed 1B v2 embedding model",\n storage=HuggingfaceStorageConfigParam(\n type="huggingface",\n # repo_id is the full model name from Hugging Face\n repo_id=HF_REPO_ID,\n repo_type="model",\n # we use the secret created in the previous step\n token_secret=hf_secret.name\n )\n )\nexcept ConflictError as e:\n print(f"Base model fileset already exists. Skipping creation.")\n base_model_fs = client.files.filesets.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\n# Create Model Entity referencing the FileSet\ntry:\n base_model = client.models.create(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n print(f"Created Model Entity: {MODEL_NAME}")\nexcept ConflictError:\n print(f"Base model already exists. Updating fileset if different.")\n base_model = client.models.update(\n workspace="default",\n name=MODEL_NAME,\n fileset=f"default/{MODEL_NAME}",\n trust_remote_code=True,\n )\n\nprint(f"\\nBase model fileset: fileset://default/{base_model.name}")\nprint("\\nBase model files:")\nprint(json.dumps([f.model_dump() for f in client.files.list(fileset=MODEL_NAME, workspace="default").data], indent=2))\n\n# Wait for ModelSpec to be populated from the checkpoint\nprint("\\nWaiting for ModelSpec to be populated...")\nSPEC_TIMEOUT_SECONDS = 120\nspec_start = time.time()\nwhile not base_model.spec:\n if time.time() - spec_start > SPEC_TIMEOUT_SECONDS:\n raise TimeoutError(f"ModelSpec not populated within {SPEC_TIMEOUT_SECONDS} seconds")\n time.sleep(2)\n base_model = client.models.retrieve(\n workspace="default",\n name=MODEL_NAME,\n )\n\nprint(f"ModelSpec populated: {base_model.spec}")\n" }, { "type": "markdown", diff --git a/docs/fern/components/notebooks/optimize-throughput.json b/docs/fern/components/notebooks/optimize-throughput.json index 452ccf038c..caa20e705d 100644 --- a/docs/fern/components/notebooks/optimize-throughput.json +++ b/docs/fern/components/notebooks/optimize-throughput.json @@ -33,9 +33,9 @@ }, { "type": "code", - "source": "%%bash\nif command -v uv >/dev/null 2>&1 && [ -n \"$VIRTUAL_ENV\" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi", + "source": "if command -v uv >/dev/null 2>&1 && [ -n \"$VIRTUAL_ENV\" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi", "language": "python", - "source_html": "%%bash\nif command -v uv >/dev/null 2>&1 && [ -n "$VIRTUAL_ENV" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi\n" + "source_html": "if command -v uv >/dev/null 2>&1 && [ -n "$VIRTUAL_ENV" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi\n" }, { "type": "markdown", diff --git a/docs/fern/components/notebooks/optimize-throughput.ts b/docs/fern/components/notebooks/optimize-throughput.ts index 59211fc597..8fab564b50 100644 --- a/docs/fern/components/notebooks/optimize-throughput.ts +++ b/docs/fern/components/notebooks/optimize-throughput.ts @@ -38,9 +38,9 @@ export default { cells: [ }, { "type": "code", - "source": "%%bash\nif command -v uv >/dev/null 2>&1 && [ -n \"$VIRTUAL_ENV\" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi", + "source": "if command -v uv >/dev/null 2>&1 && [ -n \"$VIRTUAL_ENV\" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi", "language": "python", - "source_html": "%%bash\nif command -v uv >/dev/null 2>&1 && [ -n "$VIRTUAL_ENV" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi\n" + "source_html": "if command -v uv >/dev/null 2>&1 && [ -n "$VIRTUAL_ENV" ]; then\n uv pip install datasets pandas matplotlib nvidia-ml-py\nelse\n pip install datasets pandas matplotlib nvidia-ml-py\nfi\n" }, { "type": "markdown", diff --git a/docs/fern/components/notebooks/sft-customization-job.json b/docs/fern/components/notebooks/sft-customization-job.json index 6356affdaa..287d47e3fc 100644 --- a/docs/fern/components/notebooks/sft-customization-job.json +++ b/docs/fern/components/notebooks/sft-customization-job.json @@ -177,9 +177,9 @@ }, { "type": "code", - "source": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = \"The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit.\"\nquestion = \"Who was the first person to walk on the Moon?\"\n\nmessages = [\n {\"role\": \"user\", \"content\": f\"Based on the following context, answer the question.\\n\\nContext: {context}\\n\\nQuestion: {question}\"}\n]\n\nresponse = client.inference.gateway.provider.post(\n \"v1/chat/completions\",\n name=deployment.name,\n workspace=\"default\",\n body={\n \"model\": f\"default/{OUTPUT_NAME}\",\n \"messages\": messages,\n \"temperature\": 0,\n \"max_tokens\": 128\n }\n)\n\nprint(\"=\" * 60)\nprint(\"MODEL EVALUATION\")\nprint(\"=\" * 60)\nprint(f\"Question: {question}\")\nprint(f\"Expected: Neil Armstrong\")\nprint(f\"Model output: {response['choices'][0]['message']['content']}\")", + "source": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = \"The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit.\"\nquestion = \"Who was the first person to walk on the Moon?\"\n\nprompt = f\"Context: {context} Question: {question} Answer:\"\n\nresponse = client.inference.gateway.provider.post(\n \"v1/completions\",\n name=deployment.name,\n workspace=\"default\",\n body={\n \"model\": f\"default/{OUTPUT_NAME}\",\n \"prompt\": prompt,\n \"temperature\": 0,\n \"max_tokens\": 128\n }\n)\n\nprint(\"=\" * 60)\nprint(\"MODEL EVALUATION\")\nprint(\"=\" * 60)\nprint(f\"Question: {question}\")\nprint(f\"Expected: Neil Armstrong\")\nprint(f\"Model output: {response['choices'][0]['text']}\")", "language": "python", - "source_html": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = "The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit."\nquestion = "Who was the first person to walk on the Moon?"\n\nmessages = [\n {"role": "user", "content": f"Based on the following context, answer the question.\\n\\nContext: {context}\\n\\nQuestion: {question}"}\n]\n\nresponse = client.inference.gateway.provider.post(\n "v1/chat/completions",\n name=deployment.name,\n workspace="default",\n body={\n "model": f"default/{OUTPUT_NAME}",\n "messages": messages,\n "temperature": 0,\n "max_tokens": 128\n }\n)\n\nprint("=" * 60)\nprint("MODEL EVALUATION")\nprint("=" * 60)\nprint(f"Question: {question}")\nprint(f"Expected: Neil Armstrong")\nprint(f"Model output: {response['choices'][0]['message']['content']}")\n" + "source_html": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = "The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit."\nquestion = "Who was the first person to walk on the Moon?"\n\nprompt = f"Context: {context} Question: {question} Answer:"\n\nresponse = client.inference.gateway.provider.post(\n "v1/completions",\n name=deployment.name,\n workspace="default",\n body={\n "model": f"default/{OUTPUT_NAME}",\n "prompt": prompt,\n "temperature": 0,\n "max_tokens": 128\n }\n)\n\nprint("=" * 60)\nprint("MODEL EVALUATION")\nprint("=" * 60)\nprint(f"Question: {question}")\nprint(f"Expected: Neil Armstrong")\nprint(f"Model output: {response['choices'][0]['text']}")\n" }, { "type": "markdown", diff --git a/docs/fern/components/notebooks/sft-customization-job.ts b/docs/fern/components/notebooks/sft-customization-job.ts index 850b96b06a..ea78780e27 100644 --- a/docs/fern/components/notebooks/sft-customization-job.ts +++ b/docs/fern/components/notebooks/sft-customization-job.ts @@ -182,9 +182,9 @@ export default { cells: [ }, { "type": "code", - "source": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = \"The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit.\"\nquestion = \"Who was the first person to walk on the Moon?\"\n\nmessages = [\n {\"role\": \"user\", \"content\": f\"Based on the following context, answer the question.\\n\\nContext: {context}\\n\\nQuestion: {question}\"}\n]\n\nresponse = client.inference.gateway.provider.post(\n \"v1/chat/completions\",\n name=deployment.name,\n workspace=\"default\",\n body={\n \"model\": f\"default/{OUTPUT_NAME}\",\n \"messages\": messages,\n \"temperature\": 0,\n \"max_tokens\": 128\n }\n)\n\nprint(\"=\" * 60)\nprint(\"MODEL EVALUATION\")\nprint(\"=\" * 60)\nprint(f\"Question: {question}\")\nprint(f\"Expected: Neil Armstrong\")\nprint(f\"Model output: {response['choices'][0]['message']['content']}\")", + "source": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = \"The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit.\"\nquestion = \"Who was the first person to walk on the Moon?\"\n\nprompt = f\"Context: {context} Question: {question} Answer:\"\n\nresponse = client.inference.gateway.provider.post(\n \"v1/completions\",\n name=deployment.name,\n workspace=\"default\",\n body={\n \"model\": f\"default/{OUTPUT_NAME}\",\n \"prompt\": prompt,\n \"temperature\": 0,\n \"max_tokens\": 128\n }\n)\n\nprint(\"=\" * 60)\nprint(\"MODEL EVALUATION\")\nprint(\"=\" * 60)\nprint(f\"Question: {question}\")\nprint(f\"Expected: Neil Armstrong\")\nprint(f\"Model output: {response['choices'][0]['text']}\")", "language": "python", - "source_html": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = "The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit."\nquestion = "Who was the first person to walk on the Moon?"\n\nmessages = [\n {"role": "user", "content": f"Based on the following context, answer the question.\\n\\nContext: {context}\\n\\nQuestion: {question}"}\n]\n\nresponse = client.inference.gateway.provider.post(\n "v1/chat/completions",\n name=deployment.name,\n workspace="default",\n body={\n "model": f"default/{OUTPUT_NAME}",\n "messages": messages,\n "temperature": 0,\n "max_tokens": 128\n }\n)\n\nprint("=" * 60)\nprint("MODEL EVALUATION")\nprint("=" * 60)\nprint(f"Question: {question}")\nprint(f"Expected: Neil Armstrong")\nprint(f"Model output: {response['choices'][0]['message']['content']}")\n" + "source_html": "# Wait for deployment to be ready, then test\n# Test the fine-tuned model with a question answering prompt\ncontext = "The Apollo 11 mission was the first manned mission to land on the Moon. It was launched on July 16, 1969, and Neil Armstrong became the first person to walk on the lunar surface on July 20, 1969. Buzz Aldrin joined him shortly after, while Michael Collins remained in lunar orbit."\nquestion = "Who was the first person to walk on the Moon?"\n\nprompt = f"Context: {context} Question: {question} Answer:"\n\nresponse = client.inference.gateway.provider.post(\n "v1/completions",\n name=deployment.name,\n workspace="default",\n body={\n "model": f"default/{OUTPUT_NAME}",\n "prompt": prompt,\n "temperature": 0,\n "max_tokens": 128\n }\n)\n\nprint("=" * 60)\nprint("MODEL EVALUATION")\nprint("=" * 60)\nprint(f"Question: {question}")\nprint(f"Expected: Neil Armstrong")\nprint(f"Model output: {response['choices'][0]['text']}")\n" }, { "type": "markdown",