Error: Failed to create directory "//.duckdb"!

[richb-hanover]

I cloned the prql-query repo locally, then used the Dockerfile image from ghcr.io.

From the top-level prql-query directory, I issued this command from the SQLite part of the README.md. My Google-fu doesn't find many hints for solving this.

What other debugging information could I provide? Thanks.

prql-query % pq --database sqlite://examples/chinook/sqlite/chinook.sqlite \
    'from albums | select title | take 10'
Error: IO Error: Failed to create directory "//.duckdb"!

Caused by:
    Error code 1: Unknown error code

[snth]

Thanks for the report. I've been able to reproduce the issue. (Apologies, I haven't done much testing with the docker version).

I think it is due to the user id mapping so that when pq runs it doesn't have the right permissions to write a .duckdb file (I didn't know it did that because I thought we were creating only an in-memory db).

I'll need to look into this in more detail. In the meantime a temporary workaround for you to do some testing might be changing the alias command to the following:

alias pq="docker run --rm -it -v $(pwd):/data -w /data ghcr.io/prql/prql-query:v0.0.11"

I will try to do more investigation tonight.

[richb-hanover]

That new alias command solves the problem.

I think you're right - it was caused by user mapping. The original command did something with my current userid on the host. Of course, there's no such user in the container. The new command probably works because everything runs as root.

I'm still learning the fine points of Docker, but I don't think there's any problem running as root for developer tools. I also think it would be OK to create a USER pq and let everything run as pq Thanks.

[max-sixty]

I'm still learning the fine points of Docker, but I don't think there's any problem running as root for developer tools.

I know people disagree about this, some vehemently, but this does seem like the standard. And the security context is very different from, say, running a public-facing web app in a container.

[snth]

Thanks for the feedback @richb-hanover . I have no problem with creating a pq user in the container but I don't think that would solve the problem. The reason I had the host uid and gid mappings in the alias is that so that if instead of writing to stdout you write the output to files, those files would be created with you as the owner rather than root. I still think that's the correct approach, we just have to resolve the current issue first.

---

I realised why DuckDB is trying to create that .duckdb directory - it's because because we have to install extensions at runtime. I'm working with the duckdb-rs maintainer to try and remove that (see here). Those installs are also causing the slow startup speed of pq with the DuckDB backend currently.

[snth]

Another limitation that I am aware of with the docker version of pq is that it currently doesn't pick up the PQ_* environment variables. There aren't that many of those so we could add those to the alias command as -e arguments but that doesn't seem like the most elegant solution. Any other ideas?

---

Another option for the .duckdb error would be to map /root/.local/.duckdb to the host or to a VOLUME. A volume might actually be a good option for this. I'll see if I can try this.

[max-sixty]

Good ideas @snth !

FYI if you find the docker command getting quite long with various volumes and env vars, docker-compose is a good way of having presets (it's just as good at that as it is at literally composing containers)

[snth]

That's an excellent idea @max-sixty . I've opened a separate issue #16 for that in order not to block this one.

[snth]

Fixed by #8 .