Mounting a volume from an external drive on mac

[amir-s]

Hello!

I've been trying to migrate from MySQL to PostgreSQL. The database I'm working with is quite large and I need to keep the data folder in my external SSD.

On my own laptop's drive, I can run this no problem:

docker run --rm -e POSTGRES_PASSWORD=password -v `pwd`:/var/lib/postgresql/data -p 5432:5432 postgres

It initializes the folders in the current working dir and logs out database system is ready to accept connections.

But when cwd is /Volume/ExternalSSD/something/something, it bails out, logging:

running bootstrap script ... 2020-05-08 01:53:31.550 UTC [82] FATAL:  data directory "/var/lib/postgresql/data" has invalid permissions
2020-05-08 01:53:31.550 UTC [82] DETAIL:  Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
child process exited with exit code 1

I tried --user "$(id -u):$(id -g)", didn't help. I also tried initializing the db on my local and then mv them to my external ssd, that didn't work either.

Docker volumes are not really an option here since I want the data volume to be on my external ssd.

Any help on how I can solve this would be greatly appreciated.

docker -v
Docker version 19.03.8, build afacb8b

[wglambert]

You'd find similar issues over at https://github.com/docker/for-mac/issues

There was a lot of discussion on Docker for Win/Mac volumes in #95:

MariaDB/mariadb-docker#95 (comment)

Essentially, the best solution is to put /var/lib/mysql in a volume on your Docker VM (managed via docker volume xyz commands) instead of trying to share those files directly all the way back to your Mac or Windows host filesystem.

MariaDB/mariadb-docker#95 (comment)

Yeah, just use a named volume and connect it to more than one container.

MariaDB/mariadb-docker#95 (comment)

using a named volume is basically the only option to keep the database files when running MariaDB on Docker for Windows since the host-shared folder presented to the container does not behave in a standard way.

MariaDB/mariadb-docker#95 (comment)

We recently added #168 which should make sharing a directory from a windows host possible. Try the following (I don't have a host with Docker for Windows at the moment, but it worked when I last checked):

$ # also updated to specify a version of mariadb so that things don't break when latest becomes 10.4
$ docker run --name mariadb -d -v D:\Projects\docker\docker-qub:/var/lib/mysql -e MYSQL_USER=user -e MYSQL_PASSWORD=password -e MYSQL_DATABASE=address -e >MYSQL_ROOT_PASSWORD=password mariadb:10.3 --innodb-flush-method=fsync

For future users that are using docker-compose it would be adding --innodb-flush-method=fsync to the command.

---

Most notably
MariaDB/mariadb-docker#95 (comment)

Docker and MariaDB work fine together; the bit that doesn't work is the custom filesystem employed by Docker for Windows and Docker for Mac for sharing files across the VM boundary, which is not all that surprising for a database, which often use features like mmap for performance but thus also require support from the underlying filesystem. There are many reports of similar issues with vboxsf, for example.

MariaDB/mariadb-docker#95 (comment)

Any fixes would have to happen either in Docker's shared filesystem or in MariaDB itself (not something we can really fix in this Docker image), so I'd recommend checking their respective upstream bugtrackers for any discussion of fixing the problem.

For further help you could also try asking the Docker Community Forums, Docker Community Slack, or Stack Overflow. Since these repos aren't really a user-help forum

[amir-s]

Alright, this is very helpful. Thank you @wglambert.
With MySQL I did work around the issue by passing --user argument, but I couldn't do the same with Postgres, so I thought there might be some sort of trick I could use to make this work. Since this is only for the development environment, I wouldn't object to solutions that are not "secure" or "the right way".