Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 15 additions & 6 deletions examples/experimental/swe-agent-v2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ Docker Network (swe-net)
| `swe_agent_function.py` | Custom agent function — dispatches to Harbor server, returns env metadata |
| `generate.py` | Reward function, agent metrics aggregation, `RolloutFn` |
| `download_and_process_data.py` | Download from HuggingFace or local JSONL, convert to Miles format |
| `prepare_harbor_tasks.py` | Convert Miles JSONL to Harbor task directories (generic fallback) |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The PR description mentions confirming that no other files reference prepare_harbor_tasks.py. However, there is still a stale reference to this script in the docstring of download_and_process_data.py (line 8). Please ensure this is also updated to maintain consistency across the repository.


## Step-by-Step Setup

Expand Down Expand Up @@ -117,6 +116,20 @@ pip install harbor

### Step 4: Prepare data and Harbor task directories

Harbor task directories are prepared on the agent server side using **harbor adapters**. Each adapter converts a specific dataset into Harbor's 4-file task format. For example, to prepare SWE-bench tasks:

```bash
# On the agent server (CPU machine), inside the harbor repo:
cd $CWD/harbor/adapters/swebench && uv sync

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The path $CWD/harbor-private/adapters/swebench is redundant and potentially confusing if the user is already inside the harbor-private repository as stated in the preceding comment. Additionally, $CWD is not a standard environment variable in most shells (typically $PWD is used). Using a relative path from the repository root is clearer.

Suggested change
cd $CWD/harbor/adapters/swebench && uv sync
cd adapters/swebench && uv sync


# Generate Harbor task directories for all SWE-bench Verified instances
uv run run_adapter.py --task-dir $HARBOR_TASKS_DIR --all
```

This uses the `swebench` Python package to produce correct Docker image names and Dockerfiles for each instance. Other adapters (e.g. `adapters/swe-gym`) follow the same pattern.

To prepare training data on the Miles side:

```bash
# Inside miles container:

Expand All @@ -127,10 +140,6 @@ python download_and_process_data.py --input /data/tb.jsonl --output tb.jsonl \

# Merge into one mixed JSONL
cat swe.jsonl tb.jsonl > mixed.jsonl

# Create Harbor task dirs (for custom data without a Harbor adapter)
python prepare_harbor_tasks.py --input my.jsonl --output /root/harbor_tasks/ \
--docker-network swe-net
```

Each Harbor task directory contains 4 files:
Expand Down Expand Up @@ -294,7 +303,7 @@ Agent containers need to resolve the Miles container's hostname. Ensure:

### `TaskNotFound` error

The task directory for the given `instance_id` doesn't exist under `HARBOR_TASKS_DIR`. Run the appropriate Harbor adapter or `prepare_harbor_tasks.py` first.
The task directory for the given `instance_id` doesn't exist under `HARBOR_TASKS_DIR`. Run the appropriate harbor adapter first (e.g. `adapters/swebench/run_adapter.py` for SWE-bench tasks).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While updating this section, please note that there is another stale reference to --docker-network swe-net in the troubleshooting section above (line 302 in the final file). This flag was specific to the removed prepare_harbor_tasks.py script and should be updated or removed to avoid confusion.


### SGLang engines OOM (`Not enough memory`)

Expand Down
225 changes: 0 additions & 225 deletions examples/experimental/swe-agent-v2/prepare_harbor_tasks.py

This file was deleted.

Loading