Confusing subdirectory flag behavior.

[PhilReinhold]

I'm trying to render templates from a git repository where the template is stored in a subdirectory.

I thought I might be able to use the subdirectory flag to do this, but I'm not getting any rendered output. The program exits without any output.

I would ideally like to have the copier.yml in the subdirectory, but I tried moving it to the repo root, and at this point the config questions occur, but so long as the subdirectory flag is present, no files are created.

Finally, I notice that if I omitted the subdirectory flag, and had the copier file in a subdirectory, files were created, but all of the templated values were rendered as empty strings. If config values are missing, I would expect an error rather than rendering as empty string.

[pawamoy]

Hello! It seems we simply had different use-cases in mind for a "subdirectory" feature. While you want to use the subdirectory of a repository as the whole template directory (containing copier.yml), I wanted to use the subdirectory of a repository to locate the files to generate only (copier.yml still being at the root of the repo). In fact, I wanted to mimic cookiecutter's behavior: the repo is the template, but the rendered files/dirs are inside a subdirectory.

I can totally see use-cases with both of these behaviors, for example a single repository containing multiple templates in a first layer of subdirectories, each one of them using a second layer of subdirectories to store the files/dirs to render.

repo/
    template1/
        copier.yml
        files_to_render/
    template2/
        copier.yml
        files_to_render/
    etc

[yajo]

Hi there. Thanks for your report and PR 😊

However, before merging this, I'm wondering if this is a feature we really want to support. I understand that you were expecting a different thing from the subdirectory flag, but that can be fixed better by just improving the docs.

So my question is what would be the benefit of having several templates in a single repo, vs. having one template per repo? Why do we actually need this?

I can imagine, though, some arguments in the opposite direction: why would it be better to have a single template per repo?

1. Tags. We support updating from a repo, but updates are based on tags. If I start a subproject based on your repo's template1, and then your repo evolves template2 and makes a new release, I'd have to update my subproject even if there haven't been any changes in template1.
2. Simplicity. 1 repo - 1 template. Simple. I like simple things... 🤷🏼‍♂️
3. Git submodules. If you really need to have one metarepo containing your templates, you could just have a git repo with submodules, each one pointing to another template. Then, you have all your code together, while other templates still have 1 repo each.

[balrok]

Not the author, but maybe it helps the case if you get multiple opinions. Because when I started with copier, I also missed that feature.

• At my workplace its very complicated to get a new repository (don't ask why ;-)). When I just want to quickly sketch out a copier-template I would need to apply for another repo, which usually stops me from doing so. (right now I only implemented one template exactly because of that aspect)
• Next thing is documentation: having it all in one repo allows me to write a single documentation for copier + best practices we should follow.
• Then there is consistency: our ci-pipelines/Dockerfiles are usually different but in some aspects are the same. Having to update just one repo if a central pipeline/docker-aspect changes, would safe time.

I agree, that a single repo has drawbacks especially with the "tags", but I don't think its so bad: at worst its a zero-operation I have to perform.

[yajo]

🤔 I see... I understand those problems, but still they seem like not enough to justify the need for this feature.

For instance, the problem on creating a new repo seems to be more political than technical. These days you have plenty of free pro git private hosting services that should work for almost everyone. You could still use one of these (at least from a technical point of view).

Also, centralizing docs is easy if you just put a link in the readme to your docs.

Regarding consistency, I have IMHO better plans in #286. With that fixed, you could have one metatemplate to be easily applied to your other templates.

The tags problem is a big one though. I know it's a design decision but it behaves just as expected. I think this would still overcomplicate the update logic and become at the end even more confusing than what it's trying to clarify.

[PhilReinhold]

Hello, and thank you for the prompt replies.

The first issue I was having was with the apparent failure of the subdirectory command, even with copier.yml at root. Upon further inspection, I was mistyping subdirectory name. The current behavior is to silently fail after going through the questionary, producing no output or error message. At the very least, an invalid subdirectory should throw an error.

However, there is still I would say a mismatch in expectations, and the current behavior is confusing. I was migrating from cookiecutter, where I had the repo structured like this:

/
    README.md
    template-dir/
        cookiecutter.json
        tempate_files...

I could use this repo by invoking cookiecutter <url> --directory=template-dir. If I just naively translate cookiecutter.json to copier.yml and replace --directory with subdirectory (now if i spell the subdirectory correctly) it does go ahead and run, it just skips the questionary and leaves all the template strings as blank.

I maintain that as a user I should be allowed to put my whole template in a subdirectory if I want to, but I understand the desire to not add features. If you decide not to go with this, I'd say better error reporting and documentation could have helped me. I think the following things should probably raise descriptive errors.

• Missing copier.yml
• Missing variable definitions
• Incorrect subdirectory name

And the documentation for the "--subdirectory" flag should clarify that "copier.yml" must remain at the root, and therefore that multiple templates per repo is not really supported.

[yajo]

Thanks for your reply, now we're getting somewhere 🙂

I was mistyping subdirectory name. The current behavior is to silently fail after going through the questionary, producing no output or error message. At the very least, an invalid subdirectory should throw an error.

Indeed that's a bug.

• Missing copier.yml
• Missing variable definitions

These aren't really bugs because Copier can work fine without them.

However, a missing variable should still make Jinja die while rendering. I'll double-check that case. In such case, if copier.yaml doesn't exist, the exception should indeed be more helpful on guiding the templater to create it.

And the documentation for the "--subdirectory" flag should clarify that "copier.yml" must remain at the root and therefore that multiple templates per repo is not really supported.

Also 👍 on this.

Thank you for explaining clearly your background and experience. Newcomers have a very valuable point of view, I wouldn't have realized without your help. 🙂

For you to understand this requirement: if you're a template designer and prefer to keep your template under a subdir, you need to tell that to Copier, because it is not the default behavior. To do it, you need a copier.yml with _subdirectory: example, so that file needs to be at the top level.

The CLI flag actually exists to allow an end user to choose to copy only a portion of the template. It would also let advanced templates to support copying different templates that share the same questionany. But those cases would complicate updates too if the user forgets to add the flag on every update. For example, imagine that you start a template by using no subdirectory on your template on V1, and later on V2 you wanna use one, or viceversa. Someone updating from it would fail badly if the subdirectory option were recorded as an answer. That's why that option is never recorded.

So, finally I can see that the main problem happened because of wrong expectations, which come from some background on a similar tool that can support that feature more easily because it doesn't support versioned updates. I think that another actionable fix to do is to add a migration guide for people coming from Cookiecutter.

[fdaguin]

Searching a Cookiecutter-like --directory feature in Copier, I actually end up in this discussion. Despite all the arguments previously exposed, I still believe this might be a useful feature. Furthermore, I don't think the tagging part is a problem: if I'm not mistaken, Copier doesn't enforce nor recommends any git tagging scheme (obviously, we all think about vX.X.X).

To that extent, it will be possible to use a template-name-vX.X.X scheme similar to the one used in the googleapis/repo-automation-bots repository. With some tweaks, the Copier update capabilities might be preserved.

@yajo What do you think?

[yajo]

No, sorry, the flag is already gone in master and I'm happier now. 😁

You still have the _subdirectory option in copier.yaml, and you can template its value, which should cover any use case without conflicting with updates.

Regarding tag schemes, only PEP 440 is supported. We need a standard to be able to compare things. 🤷‍♂️

Remember: 1 repo = 1 template. Simple! 🙂

[heitorlessa]

Coming in nearly a year later as I was confused about this feature (also coming from cookiecutter) -- did I understand right that I can't have a monorepo with multiple templates via --subdirectory option?

I couldn't find in the docs hence commenting in this closed issue:

repo/
    template1/
        copier.yml
        files_to_render/
    template2/
        copier.yml
        files_to_render/
    etc

[yajo]

Let's fix the confusion in #578. 🙂