Fix #1910. Add support for site aliases that point into a Docker container

[weitzman]

No description provided.

[greg-1-anderson]

Could we perhaps provide this functionality in the Site Process library, and not have to add special checking in Drush?

[greg-1-anderson]

Could we go through the exercise of defining what we would have to do to make sql:sync work with a docker-based Drupal site? What command would we use to dump the mysql database? Would we be able to do this in such a way that the target dump is written to a shared path that is readable by the local process (i.e. rsync step not needed)? What options would we need to pass to docker-compose exec to make that work? How would the sql:sync command know what to do vis-a-vis the rysnc portion of the command? What properties would we have to define in the site alias?

e.g. What would we do with this part of SqlSyncCommands::rsync?

[weitzman]

sql:sync is a good question. Assuming that rsync is out of the question, then we need to rely on Docker's array of persistence options. In general, I think folks will want to setup a Named Volume and write/read the dump file from there. sql:sync already has a --no-sync and related options which should be enough.

I may enhance our docker-compose.yml with a drupal2 service just as a POC for Docker based sql:sync.

There was some discussion in #1906 about this but no-one said how sql:sync is supposed to work across docker powered environments.

[weitzman]

The good news is that I got sql:sync to work between 2 containers who both mount a named docker volume. See last commit which contains a temporary second service and a named volume which both drupal and drupal2 mount. The command that worked is:

./drush sql:sync  @sut.local @sut.local2 --source-dump=/var/lib/sharedfiles/dump.sql --target-dump=/var/lib/sharedfiles/dump.sql.gz--no-sync

I had to specify both source and target and no-sync because of the exact lines you link to SqlSyncCommands::rsync. These paths are local but aliasRecord is considered remote (required by redispatchHook). So now we have aliases that should be "remote" for redispatchHook but local for the purposes of sql dump file paths.

I'd be OK with expecting folks to use these options, or we can refactor so that SqlSyncCommands::rsync recognizes that these are local paths. I'm not sure how to do that.

[greg-1-anderson]

Yeah, getting really close. We just need to figure out what the correct "remote" vs "not remote" determinations should be. Currently the notion of "remote" is overloaded. We need to split it up so that the key options that command line you made can be injected into the docker site alias.

[sjerdo]

What does issue #1906 have to do with this PR? I'm assuming it should be #1910.

I don't think we could assume a sync always is between two docker containers when one of the aliases uses docker for transport. And even if so, they won't share a volume just for syncing files.
Rsync could be used to sync files (or a sql dump) between containers and/or aliases (Although the local host should be temporarily used to store the file(s)).

E.g. We're currently syncing files using rsync for one of our clients using the following command using kubernetes for the remote-shell command in rsync (-e or --rsh parameter)

rsync -akz --blocking-io --rsh="kubectl exec -i project-pod-id" --delete --exclude="excluded.file.php" -- "dir/" --:"/var/www/path/to/external/dir"
or
rsync -akz --blocking-io --rsh="kubectl exec -i" --delete --exclude="excluded.file.php" dir/ project-pod-id:/var/www/path/to/external/dir

The same could be used for rsyncing files to or from a docker container:
rsync -akz --rsh="docker exec -i" --delete --exclude="excluded.file.php" dir/ container:"/var/www/path/to/external/dir"
or
rsync -akz --rsh="docker exec -i" --delete container:"/var/www/path/to/external/file.txt" file.txt

[greg-1-anderson]

We're not going to solve the sql:sync problem in this PR, but comments on the different permutations on how to handle this are appreciated. Drupal-on-Kube is another interesting use-case for sure.

[greg-1-anderson]

typo: exa,ple.site.yml

[weitzman]

@sjerdo thanks for those rsync examples. I had not seen that form before. Perhaps you could openb a PR with new Examples for the drush rsync command. Thats the best place I can think of to document this.

[greg-1-anderson]

🎉

[normanlolx]

Everybody coming here via search engine or from the other issue, you now can use the following snippet in your self.site.yml to execute Drush commands in a remote container with drush @dev cr.

dev:
  host: my.remote.host.example.com
  user: myuser
  docker:
    service: my-container-name

[normanlolx]

Sorry, last comment was probably based on a false conclusion. We had a Drush on that host itself as well as on that host in a container. I'm not sure if it's supposed to be working like that at all (as in SSH into host and execute command in a container on that host).

[normanlolx]

@weitzman – Do I understand this right, the Docker support is not meant to be used to connect to a remote host that's running Docker containers, and to exec Drush then inside this remote's host containers? It's meant to be used to execute Drush commands in a respectively local Docker container?

[weitzman]

@leymannx - my understanding is that your scenario works fine. This is an abstraction provided by Docker itself. Just set DOCKER_HOST env variable to point to the remote docker.

[csheinin]

Soo.... here's what I'm deriving from this for mysitename.site.yml:

local:
  docker:
    service: drupal
    exec:
      options: --user root
  uri: dev
  root: /var/www/html/web
  paths:
    drush-script: /var/www/html/drush

Does this imply we should make sure we have drush installed inside the docker instance itself? Does anyone have time to jot down some notes of "what is required so that drush pm-list command from my local box returns output from target drupal-in-container"?

[Trogie-Vynckier]

Nice! Any idea how to combine ssh to a machine that runs drupal/drush in a container?

[lukasfischer]

As I understand, th use case ssh to remote + entering docker container on remote server is not supported. I created the following bash script - it assumes you use drupal with drush and lando:

ssh -t $DOCKER_PROD_USER@$DOCKER_PROD_HOST "cd $DOCKER_PROD_DOCKER_PATH; exec docker compose exec -T drupal sh -c 'drush sql-dump --structure-tables-list=cache,cache_*,search_index,watchdog'" > dump.sql lando db-import dump.sql rm dump.sql

If there is a more elegant solution using drush site alias, I'm curious.

fyi @Trogie-Vynckier