Please add support for SOURCE_DATE_EPOCH to enable reproducible builds

[alerque]

Currently this library is blocking every project that uses it from passing reproducible build checks. The primary offender is here:

shadow-rs/src/env.rs

Line 253 in 1e9feec

let time = Local::now();

The standardized solution to this it to support SOURCE_DATE_EPOCH. This way distro build systems and anything that is trying to verify whether packages are reproducible can keep ephemeral time information from being embedded in the binaries during test builds.

As an example of where this is a problem, see projects such as starship that are showing up in the BAD list on Arch Linux's reproducible status dashboard. The relevant bit from the diffoscope output is this snippet:

│ │ ├── strings --all --bytes=8 {}
│ │ │ @@ -5266,15 +5266,15 @@
│ │ │  - Build Time: 
│ │ │  #### Relevant Shell Configuration
│ │ │  #### Starship Configuration
│ │ │  releasestarship::utilssrc/utils.rsExecuting command  failed by: You can set command_timeout in your config to a higher value to allow longer-running commands to keep executing. timed out., stderr: , exit code: "Unable to decode stderr: Unable to decode stdout: Unable to run Unable to find  in PATH, Using  with args --version
│ │ │  pkg_version:
│ │ │  commit_hash:
│ │ │  build_time:
│ │ │ -build_env:,rustc 1.53.02021-07-18 19:25:400.56.0fg:bg:redsrc/config.rsCould not parse color in string: Read predefined color: Read ANSI color string: Read RGB color string: Could not parse hexadecimal string: Attempting to read hexadecimal color string: Parsing color_string: /build/rust/src/rustc-1.53.0-src/library/std/src/thread/local.rsFile read sucessfullyError reading file: ErrOkTrying to read from Unable to parse the config file: Config parsed: assertion failed: self.next_value.is_none()Unable to read table valuesassertion failed: self.values.next().is_none()attempt to join into collection with len > usize::MAX/build/rust/src/rustc-1.53.0-src/library/alloc/src/str.rs$__toml_private_datetimeUnable to read config file content: Config file content: "
│ │ │ +build_env:,rustc 1.53.02021-08-02 14:35:450.56.0fg:bg:redsrc/config.rsCould not parse color in string: Read predefined color: Read ANSI color string: Read RGB color string: Could not parse hexadecimal string: Attempting to read hexadecimal color string: Parsing color_string: /build/rust/src/rustc-1.53.0-src/library/std/src/thread/local.rsFile read sucessfullyError reading file: ErrOkTrying to read from Unable to parse the config file: Config parsed: assertion failed: self.next_value.is_none()Unable to read table valuesassertion failed: self.values.next().is_none()attempt to join into collection with len > usize::MAX/build/rust/src/rustc-1.53.0-src/library/alloc/src/str.rs$__toml_private_datetimeUnable to read config file content: Config file content: "
│ │ │  Using default config path: STARSHIP_CONFIG is not setSTARSHIP_CONFIG is set: Config found for "": No config found for "": Option "" not foundStarship::get_config called with an empty pathcustomCustom config found for "env_varformatscan_timeoutcommand_timeoutadd_newlinesrc/configs/starship_root.rsDid you mean ''?Unknown config key 'awsbatterycharactercmakecmd_durationcondacrystaldirectorydocker_contextdotnetelixirelmerlanggcloudgit_branchgit_commitgit_metricsgit_stategit_statusgolanghg_branchjuliakotlinkubernetesline_breakluamemory_usagenimnix_shellnodejsocamlopenstackpackagephppurescriptpythonrlangscalashellshlvlsingularityswiftterraformvagrantvlangzigon [$symbol($profile )(\($region\) )(\[$duration\])]($style)bold yellowX[$symbol$percentage]($style) [
│ │ │  ](bold green)[
│ │ │  ](bold red)[
│ │ │  ](bold green)CMakeLists.txtCMakeCache.txtv${raw}bold bluetook [$duration]($style) yellow boldvia [$symbol$environment]($style) 
│ │ │   green boldcrshard.yml
│ │ │   [$symbol($output )]($style)<custom config>pubspec.yamlpubspec.ymlpubspec.lock.dart_tool
│ │ │   mod.tsdeps.tsmod.jsdeps.js
│ │ ├── readelf --wide --decompress --hex-dump=.rodata {}
│ │ │ @@ -27659,16 +27659,16 @@
│ │ │    0x0048c080 66696e64 2020696e 20504154 482c2055 find  in PATH, U
│ │ │    0x0048c090 73696e67 20207769 74682061 72677320 sing  with args 
│ │ │    0x0048c0a0 2d2d7665 7273696f 6e0a0a0a 706b675f --version...pkg_
│ │ │    0x0048c0b0 76657273 696f6e3a 0a746167 3a0a636f version:.tag:.co
│ │ │    0x0048c0c0 6d6d6974 5f686173 683a0a62 75696c64 mmit_hash:.build
│ │ │    0x0048c0d0 5f74696d 653a0a62 75696c64 5f656e76 _time:.build_env
│ │ │    0x0048c0e0 3a2c7275 73746320 312e3533 2e303230 :,rustc 1.53.020
│ │ │ -  0x0048c0f0 32312d30 372d3138 2031393a 32353a34 21-07-18 19:25:4
│ │ │ -  0x0048c100 30302e35 362e3066 673a6267 3a726564 00.56.0fg:bg:red
│ │ │ +  0x0048c0f0 32312d30 382d3032 2031343a 33353a34 21-08-02 14:35:4
│ │ │ +  0x0048c100 35302e35 362e3066 673a6267 3a726564 50.56.0fg:bg:red
│ │ │    0x0048c110 7372632f 636f6e66 69672e72 73436f75 src/config.rsCou
│ │ │    0x0048c120 6c64206e 6f742070 61727365 20636f6c ld not parse col
│ │ │    0x0048c130 6f722069 6e207374 72696e67 3a205265 or in string: Re
│ │ │    0x0048c140 61642070 72656465 66696e65 6420636f ad predefined co
│ │ │    0x0048c150 6c6f723a 20526561 6420414e 53492063 lor: Read ANSI c
│ │ │    0x0048c160 6f6c6f72 20737472 696e673a 20526561 olor string: Rea
│ │ │    0x0048c170 64205247 4220636f 6c6f7220 73747269 d RGB color stri

[baoyachi]

@alerque Is there any example support?

[baoyachi]

add unit test:

shadow-rs/src/env.rs

Lines 384 to 390 in 479ae8f

	#[test]
	fn test_now_data_time() {
	std::env::set_var("SOURCE_DATE_EPOCH", "1628080443");
	let time = now_data_time();
	let now = Local::now();
	assert!(time < now);
	}

[baoyachi]

@alerque Thanks for your mr.

It's seem use new shadow-rs version can solve it.

shadow-rs = "0.6.5"

[anthraxx]

@baoyachi that looks like a partially good way to handle it. However, it is still using Local, which means that, when interacting with the object, it will depend on the TZ timezone value. Hence while SOURCE_DATE_EPOCH has the same value, the resulting string via time.format will yield different results depending on the system timezone.
I believe you need to simply substitute Local with Utc and that should solve it.

[alerque]

@anthraxx Is that in reference the test above (which does use a local time zone but doesn't matter because it only tests that the local time is different that a fixed time passed with the end var) or do you think my actual implementation is a problem? In the actual implementation I'm using Local.timestamp(epoch, 0) where epoch is the parsed env var value and 0 is a timezone (UTC). Hence it should always come out with the same time, no?

[anthraxx]

@alerque yes, the issue is with the actual implementation. The DateTime struct is initialized with a UTC epoch, but for output operations it still encodes the system locale's timezone setting.

    let time = Local.timestamp(1628685351, 0);
    println!("{}", time.format("%Y-%m-%d %H:%M:%S"));
    let time = Utc.timestamp(1628685351, 0);
    println!("{}", time.format("%Y-%m-%d %H:%M:%S"));

TZ="Europe/Berlin" cargo run
2021-08-11 14:35:51
2021-08-11 12:35:51

TZ="UTC" cargo run
2021-08-11 12:35:51
2021-08-11 12:35:51

[baoyachi]

I'll optimize it.

[alerque]

I see, good catch and my apologies for the incomplete implementation.

@baoyachi I'm happy to setup a PR if you want but it looks like it might be just as easy for you to make the correction. Let me know...

[baoyachi]

Wait a minute. There will be a new version after modification 🚀

[baoyachi]

@alerque @anthraxx

I'm sorry about this. I lost TimeZone when I was formatting the time.
Then I add the TimeZone to the format.

https://github.com/baoyachi/shadow-rs/blob/feature/issue_61/src/time.rs#L23

[baoyachi]

new version published

shadow-rs = "0.6.6"