Skip to content

v4.0.x: reproducible builds + portability fix #8147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
gpaulsen merged 4 commits into from
Nov 4, 2020
Merged

v4.0.x: reproducible builds + portability fix #8147

gpaulsen merged 4 commits into from
Nov 4, 2020

Conversation

@jsquyres
Copy link
Member

@jsquyres jsquyres commented Oct 28, 2020 •
edited
Loading

Cherry pick of #8136 and #8152 from master regarding reproducible builds. Thanks to @bmwiedemann for bringing this back to our attention.

Also bring in a MacOS/readlink portability fix.

@jsquyres
opal_functions.m4: remove redundant code
477266b 
This code was invoked twice.  Leave it solely in OPAL_CONFIGURE_SETUP,
which is invoked before OPAL_BASIC_SETUP.

Signed-off-by: Jeff Squyres <[email protected]>
(cherry picked from commit 7c36b45)
@jsquyres jsquyres added this to the v4.0.6 milestone Oct 28, 2020
@bmwiedemann
Copy link
Contributor

bmwiedemann commented Oct 28, 2020

This and 4.1 needs one more patch:

orte/tools/orte-info/Makefile.am
-            -DOMPI_BUILD_DATE="\"`date`\"" \
+            -DOMPI_BUILD_DATE="\"`$(top_srcdir)/config/getdate.sh`\"" \

@jsquyres
Copy link
Member Author

jsquyres commented Oct 28, 2020

@bmwiedemann I notice on master that in one place, we have:

${USER:-`whoami`}

but in other places we just have

$$USER

Should they all be of the form

# With $ or $$, as appropriate
{USER:-`whoami`}

?

@bmwiedemann
Copy link
Contributor

bmwiedemann commented Oct 28, 2020

I'd leave it as is. I have seen good PRs with feature-creep in the past that got stalled too long.

@jsquyres
Copy link
Member Author

jsquyres commented Oct 28, 2020

Ah, I see -- $USER is an automatically-defined shell variable, anyway (I thought it was something unique to Reproducible builds, and therefore might not actually be set).

@rhc54
Copy link
Contributor

rhc54 commented Oct 28, 2020

Is it safe to say we need similar changes in PMIx and PRRTE?

@bmwiedemann
Copy link
Contributor

bmwiedemann commented Oct 28, 2020

@rhc54 I tested with our openmpi4 package and these changes (with orte-info) were enough to make it reproducible.

@rhc54
Copy link
Contributor

rhc54 commented Oct 28, 2020

Understood - my apologies. My question was more about building PMIx and PRRTE standalone packages (i.e., when they aren't embedded in OMPI)

@jsquyres
Copy link
Member Author

jsquyres commented Oct 28, 2020

@rhc54 It would not be a bad thing to also apply this to PMIx or PRRTE. I wouldn't say it's critical, but it's definitely in the "nice to have" category.

@jsquyres jsquyres force-pushed the pr/v4.0.x/reproducible-build branch from 4c95519 to 680f89f Compare October 28, 2020 19:44
@jsquyres
Copy link
Member Author

jsquyres commented Oct 29, 2020

This PR will also need the MacOS portability fixes from #8152.

@jsquyres
Copy link
Member Author

jsquyres commented Oct 29, 2020

bot:ompi:retest

@jsquyres jsquyres changed the title v4.0.x: reproducible builds v4.0.x: reproducible builds + portability fix Oct 29, 2020
@jsquyres jsquyres added the NEWS label Oct 29, 2020
bmwiedemann
bmwiedemann reviewed Oct 29, 2020
View reviewed changes
ggouaillardet and others added 3 commits October 29, 2020 08:14
@ggouaillardet @jsquyres
configury: make build Reproducible
456d164 
If defined, use SOURCE_DATE_EPOCH environment variable; make the build
Reproducible by forcing timestamps.  See
https://reproducible-builds.org/docs/source-date-epoch/ for more
information.

Thanks Bernhard M. Wiedemann for bringing this to our attention.

Fixes open-mpi#3759

**NOTE:** This was cherry-picked from master, and slightly modified /
  amended for the v4.0.x branch.

Signed-off-by: Gilles Gouaillardet <[email protected]>
Signed-off-by: Bernhard M. Wiedemann <[email protected]>
Signed-off-by: Jeff Squyres <[email protected]>
(cherry picked from commit 7b4e8ba)
@jsquyres
configure.ac: Add workaround on MacOS for "readlink -f"
a5748b4 
MacOS does not have "readlink -f" or "realpath", so use the
MacOS-provided Python, which we know has os.path.realpath().

Signed-off-by: Jeff Squyres <[email protected]>
(cherry picked from commit ddf216b)
@jsquyres
getdate.sh: make the date(1) usage more portable
8f73cb1 
There are several different flavors of date(1) out there.  Try a few
different CLI options for date(1) to see which one works.

Signed-off-by: Jeff Squyres <[email protected]>
(cherry picked from commit 89920ba)
@jsquyres jsquyres force-pushed the pr/v4.0.x/reproducible-build branch from 8886c89 to 8f73cb1 Compare October 29, 2020 15:15
@gpaulsen gpaulsen merged commit 14d7da1 into open-mpi:v4.0.x Nov 4, 2020
@jsquyres jsquyres deleted the pr/v4.0.x/reproducible-build branch July 17, 2022 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rhc54 rhc54 rhc54 approved these changes

@ggouaillardet ggouaillardet Awaiting requested review from ggouaillardet

+1 more reviewer

@bmwiedemann bmwiedemann bmwiedemann approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

enhancement NEWS Target: v4.0.x

Projects

None yet

Milestone

v4.0.6

Development

Successfully merging this pull request may close these issues.

5 participants

@jsquyres @bmwiedemann @rhc54 @gpaulsen @ggouaillardet