Skip to content

feat(docker-compose): Add Docker Compose stack that mounts a local directory. #440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sitaowang1998 merged 14 commits into from
Aug 20, 2026
Merged

feat(docker-compose): Add Docker Compose stack that mounts a local directory. #440

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
da04bff
Add the docker compose file and task to start a cluster and mount a f…
sitaowang1998 Aug 14, 2026
d5f7d03
Add env for image refs
sitaowang1998 Aug 14, 2026
8213b8b
Resolve relative path to full path
sitaowang1998 Aug 14, 2026
f0fb689
Add image ref for env
sitaowang1998 Aug 18, 2026
f40d736
Merge branch 'main' into docker-compose-local
sitaowang1998 Aug 18, 2026
ed735a3
Add build and up task
sitaowang1998 Aug 18, 2026
02f94e5
Merge branch 'main' into docker-compose-local
sitaowang1998 Aug 18, 2026
672c58d
Merge branch 'main' into docker-compose-local
sitaowang1998 Aug 18, 2026
8daa1fd
Fix name
sitaowang1998 Aug 19, 2026
e7a6215
Apply suggestions from code review
sitaowang1998 Aug 19, 2026
65601b9
Fix image pull policy
sitaowang1998 Aug 19, 2026
74d84ef
Merge branch 'main' into docker-compose-local
sitaowang1998 Aug 19, 2026
5d02d68
Merge branch 'main' into docker-compose-local
sitaowang1998 Aug 20, 2026
b640495
Remove build-and-up
sitaowang1998 Aug 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions taskfiles/docker.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,26 @@ tasks:
compose:clean:
dir: "{{.G_SPIDER_COMPOSE_DIR}}"
cmd: "docker compose --file compose.yaml down --volumes --remove-orphans"

# Starts a local Spider deployment with libraries mounted into every worker.

@20001020ycx 20001020ycx Aug 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add image override to the docstring

@sitaowang1998 sitaowang1998 Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are already listed as parameters.

@20001020ycx 20001020ycx Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my latest review comment, that's what I mean by this

Comment thread
sitaowang1998 marked this conversation as resolved.
Resolved
#
# @param {string} SPIDER_WORKER_LIBRARY_DIR The host directory containing the libraries.
Comment thread
20001020ycx marked this conversation as resolved.
Resolved
compose:local:up:
env:
SPIDER_WORKER_LIBRARY_DIR: "{{.SPIDER_WORKER_LIBRARY_DIR}}"
requires:
vars: ["SPIDER_WORKER_LIBRARY_DIR"]
dir: "{{.G_SPIDER_COMPOSE_DIR}}"
cmd: >-
docker compose --project-name spider-local --file compose.yaml
--file compose.local.yaml up --detach --wait --wait-timeout 300
Comment thread
20001020ycx marked this conversation as resolved.
Resolved

compose:local:down:
dir: "{{.G_SPIDER_COMPOSE_DIR}}"
cmd: "docker compose --project-name spider-local --file compose.yaml down"

compose:local:clean:
dir: "{{.G_SPIDER_COMPOSE_DIR}}"
cmd: >-
docker compose --project-name spider-local --file compose.yaml down
--volumes --remove-orphans
7 changes: 7 additions & 0 deletions tools/deployment/spider-compose/compose.local.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
services:
spider-worker:
volumes:
- type: "bind"
source: "${SPIDER_WORKER_LIBRARY_DIR:?}"
target: "${SPIDER_WORKER_PACKAGE_DIR:-/opt/spider/packages}"
read_only: true
Loading