Handling of repo-level README.md

[sbidoul]

When this project was initialized back in May, copier did not support rich text answers. So the description field in README.md was limited to a single line. Some repo currently have richer descriptions (bullet points, links).

On the other hand, managing README.md via the template has huge benefits in terms of harmonizing the files across repo and facilitating updates.

@yajo do you know if copier now support multiline text answers?

Irrespective of the answer, I'm currently leaning towards creating fresh README.md from the template for the 14.0 branch, and then let PSC update the answers file to provide the content.

What do people think?

[yajo]

I'm drafting copier-org/copier#260 but it's not ready yet. But indeed it's my planned main next enhancement.

It's sad that there's no interactive support for multiline answers, but currently there's a workaround:

multiline_answer='this is

my multiline answer'

copier -d "multiline_question=$multiline_answer" [...]

I hope that unblocks you for the time being.

[dreispt]

@sbidoul There is already automation to generate the addons table in the repo README.
What if that automation used a text fragment for the repo description, similarly to what we do now on module READMEs?
If so, this would not be a concern for copier.

[yajo]

I think copier is a better candidate here, since you're not changing descriptions all the time. But yes, it's an option.

[sbidoul]

@yajo I have difficulties using copier update with README.md.

What I'm trying to do is this, as it is what we'd expect PSC to do to update their README:

• update and git commit .copier-answers.yml with a new repo_description and repo_name (this should be done by repo maintainer)

• running copier --vcs-ref=HEAD --force update says it updates README.md but apparently does nothing (this is what the bot would do, periodically, or maybe when detecting a change to .copier-answers.yml)

Either c383d11 or HEAD is not a PEP 440 valid version.

 identical  .editorconfig
 identical  .eslintrc.yml
 identical  .flake8
 identical  .gitignore
 identical  .isort.cfg
 identical  .pre-commit-config.yaml
 identical  .prettierrc.yml
 identical  .pylintrc-mandatory
 identical  .pylintrc
 identical  .travis.yml
 identical  CONTRIBUTING.md
 identical  LICENSE
  conflict  README.md
     force  README.md
 identical  oca_dependencies.txt
 identical  .copier-answers.yml

But README.md stays unchanched. I'm using copier 5.1.

[yajo]

Well, this is not working for several reasons, but don't worry. Let me teach you a bit how Copier works.

First of all, go read https://copier.readthedocs.io/en/latest/updating/ carefully (these docs are from master, which have improved recently this specific part).

So, the 1st problem:

Either c383d11 or HEAD is not a PEP 440 valid version.

For copier update to work reliably, it needs the template to be versioned using git tags. I'll release v0.1.0 after #8 is merged, and you can start testing. Or you can just tag it locally and use the local template to test.

Before you ask, there is support for prereleases, so when planning a new release, let's say v1.0.0, if we want to test, it's as simple as publishing an alpha release named v1.0.0a0 (for example). This will make downstream users of the template ignore this tag, unless explicitly passed --prerelease, and we will be able to test updates, including migrations.

I can publish the 1st alpha now if you want.

Second problem:

update and git commit .copier-answers.yml with a new repo_description and repo_name (this should be done by repo maintainer)

This is not how copier works.

Copier expects that a given set of answers (which include user-provided answers and copier stuff like _commit) produces a given project. If you alter .copier-answers.yml manually, you are tricking Copier, making it assume that those new answers produced the README.md file you have right now. That's a lie, so you cannot expect updates to work properly.

The process to update answers is to use copier itself. Example:

copier --data project_description='My new description.'

I thought it was explained here, but I'll explain it better for the wider audience (see #11) that doesn't know Copier very well:

oca-addons-repo-template/{{ _copier_conf.answers_file }}.jinja

Line 1 in 3fa9c39

# Changes here will be overwritten by Copier

running copier --vcs-ref=HEAD --force update says it updates README.md but apparently does nothing (this is what the bot would do, periodically, or maybe when detecting a change to .copier-answers.yml)

As you probably have guessed by now, this assumption should change a bit:

1. You wouldn't need --vcs-ref=HEAD. Actually that would produce bugs, because copier updates are based on releases, not commits. This is important to prevent downgrades and to apply migrations.
2. Because of that, the bot should update OCA repos only when detecting a new release in the template repo, and not when doing a commit. BTW this can be automated very easily with github actions, so maybe you don't even need the bot, although that's another subject I guess.
3. The changes should never be done manually to .copier-answers.yml.
4. Finally, the bot doesn't need to call copier using its CLI. It can be called by API too if the bot runs in Python 3.6.1+.

A possible workflow would be:

1. oca-addons-repo-template gets a release.
2. A GH action here notifies OCA bot.
3. The bot applies the update in all OCA repos:
   1. Run copier -f
   2. Run pre-commit run -a
   3. Commit.
   4. Open PR.
   5. Set to automerge, probably.

Another possible workflow would be the same as the above, but skipping steps 1 and 2, and going directly to a scheduled update like outlined in step 3. After all, it should produce no diff if the template version is the same.

[sbidoul]

Thanks for the explanation, @yajo it was much needed apparently. And I did RTFM.

I'm not sure my issues are related to the absence of tags in the template as what I was trying to do is mainly update the README without changing the template.

copier --data project_description='My new description.'

So where are we going to get My new description from if it can't be .copier-answers.yml ?

[yajo]

Happy to help! ☺️

So where are we going to get My new description from if it can't be .copier-answers.yml ?

Well, you can get it from there. You just don't write it there. You tell copier the new answer, and then Copier will write it there, at the same time as it updates the README.

[sbidoul]

I don't understand. I'm going to generate the 14.0 branches with basically empty README.md. Then we need to ask PSCs to put their title and description somewhere to update the README.md. How do we do that?

[sbidoul]

And if we need another place than .copier-answers.yml we'll get the same information twice in the repo. @yajo
I don't really understand why .copier-answers.yml is not considered by copier as the input for answering the questions automatically.

[sbidoul]

@yajo Or do you think we can simply let users modify README.md and let copier' smart diff algorithm do the job?

In that case, maybe we should remove the questions for the title and description, and put comments to mark the sections that the user should not change manually?

But then would remain the problem of preserving the README content across Odoo series.

[sbidoul]

But then would remain the problem of preserving the README content across Odoo series.

We'll see how to address that next year. For now let's go with what we have here, i.e. letting PSC update README.md manually and copier' smart diff mechanism do what it can to preserve it.

[yajo]

I don't really understand why .copier-answers.yml is not considered by copier as the input for answering the questions automatically.

I explained that above in #6 (comment), but let me explain again differently.

Imagine this .copier-answers.yml:

_commit: v1.0.0
_src_path: https://github.com/OCA/oca-addons-repo-template.git
description: This is my nice description

Then, imagine this template README.md.jinja:

# Sample README

{{ description }}

It would produce this README.md file:

# Sample README

This is my nice description

OK, up until now all is well understood, right?

Now, imagine you go and edit .copier-answers.yml directly and change this:

-description: This is my nice description
+description: Updated description

And then git commit -am 'change description badly'.

Now, you run copier --force update. Do you expect this to update the template? It won't happen because this is how Copier thinks:

🤖 💭 OK, so when the user answered Updated description the last time, then the resulting README.md is:

# Sample README

This is my nice description

🤖 💭 Then, what would result with current answers? Let me check... Well, current answer is also Updated description, so README.md shouldn't change. Resulting README.md is again:

# Sample README

This is my nice description

See? You tricked Copier and shot yourself in the foot. 🦶 🔫

---

The correct process is:

You still have this original .copier-answers.yml:

Imagine this .copier-answers.yml:

_commit: v1.0.0
_src_path: https://github.com/OCA/oca-addons-repo-template.git
description: This is my nice description

Then you ask copier to update the template with different answers:

copier --data description='Updated description' update

Then Copier thinks:

🤖 💭 OK, so when the user answered This is my nice description the last time, then the resulting README.md is:

# Sample README

This is my nice description

🤖 💭 Then, what would result with current answers? Let me check... The current answer has changed to Updated description, so README.md should change. Resulting README.md is updated to:

# Sample README

Updated description

Everyone's happy. 😊

[yajo]

But then would remain the problem of preserving the README content across Odoo series.

Oh I forgot this part.

#12 is not wrong per se, it's just another approach that will work perfectly. However, indeed, it would be better to include a new question in the form instead of relying on manual modifications because of this detail.

Just remember: never edit .copier-answers.yml by hand. Instead, give answers to Copier and let it write on that file.