Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-3.5] scripts: Avoid additional repo clone #14050

Merged
merged 2 commits into from
May 18, 2022

Conversation

serathius
Copy link
Member

This PR removes additional clone when building artifacts.

When releasing v3.5.4 this clone was main cause of issues and
confusion about what release script is doing.

release.sh script already clones repo in /tmp/ directory, so clonning
before build is not needed. As precautions for bug in script leaving
/tmp/ clone in bad state I moved "Verify the latest commit has the
version tag" and added "Verify the clean working tree" to be always run
before build.

cc @ahrtr @ptabor

This PR removes additional clone when building artifacts.

When releasing v3.5.4 this clone was main cause of issues and
confusion about what release script is doing.

release.sh script already clones repo in /tmp/ directory, so clonning
before build is not needed. As precautions for bug in script leaving
/tmp/ clone in bad state  I moved "Verify the latest commit has the
version tag" and added "Verify the clean working tree" to be always run
before build.
scripts/release Outdated

# Verify the clean working tree
# shellcheck disable=SC2155
local diff="$(git diff --stat)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might miss 2 types of the changes:
1 changes already staged for commit (i.e. git diff --cached)
2. .gitigored files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should check for 1.
I think we can accept 2. as it's a minor risk to be source of inconsistency.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used git diff HEAD to catch both stages and unstages changes.
For 2 it would be worrying that files listed in gitignore could impact built release .

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(no HEAD) yet here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:P added #14051 on top

Copy link
Member

@ahrtr ahrtr May 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, we might need to use command below. It can cover both unstaged and staged changes,

git status --short

scripts/release Outdated Show resolved Hide resolved
@ptabor
Copy link
Contributor

ptabor commented May 18, 2022

If think that the 'clone from remote' desire was to be 200% sure that the build is 'repeatable', i.e.
whenever user will checkout the tagged branch themselves and call build-release.sh will get exact same artifacts.

Overall SG. But suggested 2 fixes - including one additional safe-guard (no --staged changes).

@ptabor
Copy link
Contributor

ptabor commented May 18, 2022

Was it already submitted to main or for main there is another approach ?

Edit: I see - it's cherry pick of: #14044. Please call it explicitly in the PR description and pls. consider proposed changes for main as well.

@serathius
Copy link
Member Author

Will send fix to main first

@serathius
Copy link
Member Author

If think that the 'clone from remote' desire was to be 200% sure that the build is 'repeatable', i.e. whenever user will checkout the tagged branch themselves and call build-release.sh will get exact same artifacts.

The main problem was that it made call execution untracable. This is overall problem with our scripts, sometimes we source based on local path source ./scripts/test_lib.sh, sometimes we call scripts ./scripts/build.sh and sometimes we change directories to clones REPOSITORY=$(pwd) ./scripts/build-release.sh. All the cases mentioned are mixed during execution, so when there is an bug it's almost impossible to find which code is broken.

If we are worried about side effects from building, we should rather invest time into making sure that our scripts are not producing them. I'm always sad when I need to periodically run find . -name '*.log' | xargs rm (and many other extensions) on local etcd repo to clean up random files created when running tests.

@serathius serathius merged commit 73876b1 into etcd-io:release-3.5 May 18, 2022
@serathius serathius deleted the avoid-clone-v3.5 branch June 15, 2023 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants