v2.8.28's .tgz contains corrupt timestamps, causing errors from zip command and others

[jesstelford]

Bug report or feature request?

Bug report

Uglify version (uglifyjs -V)

$ ./node_modules/.bin/uglifyjs -V
uglify-js 2.8.28

Steps to reproduce

$ npm init -y
$ npm install [email protected]
$ ls node_modules/uglify-js
total 104
-rw-r--r--   1 jess.telford  staff   1.3K  1 Jan  1970 LICENSE
-rw-r--r--   1 jess.telford  staff    41K  1 Jan  1970 README.md
...

Notice the Jan 1st, 1970 timestamp.

This is causing issues on certain platforms where the timestamp is actually reported as December 31st, 1969, causing POSIX tools to fail. (ie; it's a -1 unix timestamp).

Edit: The actual error is: ZIP does not support timestamps before 1980 on Ubuntu 14.04 (our build machine)

Analysis

It looks like it's the actual files inside the package, which implies it's not anything to do with npm, but instead to do with the state of the filesystem at the time of publish Edit: It was npm! (I've accidentally caused bad publishes like this myself in the past):

$ curl https://registry.npmjs.org/uglify-js/-/uglify-js-2.8.28.tgz | tar -tvf -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0-rw-rw-rw-  0 0      0        1139  1 Jan  1970 package/package.json
-rw-rw-rw-  0 0      0       41915  1 Jan  1970 package/README.md
-rw-rw-rw-  0 0      0        1348  1 Jan  1970 package/LICENSE
-rw-rw-rw-  0 0      0        1982  1 Jan  1970 package/bin/extract-props.js
-rw-rw-rw-  0 0      0       21426  1 Jan  1970 package/bin/uglifyjs
-rw-rw-rw-  0 0      0       11096  1 Jan  1970 package/lib/utils.js
-rw-rw-rw-  0 0      0        6938  1 Jan  1970 package/lib/transform.js
-rw-rw-rw-  0 0      0        3644  1 Jan  1970 package/lib/sourcemap.js
-rw-rw-rw-  0 0      0       23483  1 Jan  1970 package/lib/scope.js
-rw-rw-rw-  0 0      0        8895  1 Jan  1970 package/lib/propmangle.js
-rw-rw-rw-  0 0      0       57351  1 Jan  1970 package/lib/parse.js
-rw-rw-rw-  0 0      0       47999  1 Jan  1970 package/lib/output.js
-rw-rw-rw-  0 0      0       22265  1 Jan  1970 package/lib/mozilla-ast.js
-rw-rw-rw-  0 0      0      179487  1 Jan  1970 package/lib/compress.js
-rw-rw-rw-  0 0      0       34888  1 Jan  1970 package/lib/ast.js
-rw-rw-rw-  0 0      0       10194  1 Jan  1970 package/tools/node.js
-rw-rw-rw-  0 0      0         688  1 Jan  1970 package/tools/exports.js
100  126k  100  126k    0     0   232k      0 --:--:-- --:--:-- --:--:--  232k

-rw-rw-rw-  0 0      0        1640  1 Jan  1970 package/tools/props.html

Note that this is not a problem in 2.8.27:

$ curl https://registry.npmjs.org/uglify-js/-/uglify-js-2.8.27.tgz | tar -tvf -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0-rw-rw-rw-  0 0      0        1139 19 May 19:53 package/package.json
-rw-rw-rw-  0 0      0       41915 19 May 19:53 package/README.md
-rw-rw-rw-  0 0      0        1348 15 Jan 21:32 package/LICENSE
-rw-rw-rw-  0 0      0        1982 19 May 19:53 package/bin/extract-props.js
-rw-rw-rw-  0 0      0       21426 19 May 19:53 package/bin/uglifyjs
-rw-rw-rw-  0 0      0       34888 19 May 19:53 package/lib/ast.js
-rw-rw-rw-  0 0      0      179487 19 May 19:53 package/lib/compress.js
-rw-rw-rw-  0 0      0       22265 19 May 19:53 package/lib/mozilla-ast.js
-rw-rw-rw-  0 0      0       47845 19 May 19:53 package/lib/output.js
-rw-rw-rw-  0 0      0       57351 19 May 19:53 package/lib/parse.js
-rw-rw-rw-  0 0      0        8895 19 May 19:53 package/lib/propmangle.js
-rw-rw-rw-  0 0      0       23483 19 May 19:53 package/lib/scope.js
-rw-rw-rw-  0 0      0        3644 22 Mar 22:18 package/lib/sourcemap.js
100  127k  100  127k    0     0   132k      0 --:--:-- --:--:-- --:--:--  132k

-rw-rw-rw-  0 0      0       11096 19 May 19:53 package/lib/utils.js
-rw-rw-rw-  0 0      0         688 19 May 19:53 package/tools/exports.js
-rw-rw-rw-  0 0      0       10194 19 May 19:53 package/tools/node.js
-rw-rw-rw-  0 0      0      142838 19 May 19:53 package/tools/domprops.json
-rw-rw-rw-  0 0      0        1640 15 Jan 21:32 package/tools/props.html

Edit: Related to #2054

Edit 2: here's an incomplete list of other issues that appear to have a root cause as outlined above:

• Code Deploy - Unhandled exception - ZIP does not support timestamps before 1980  aws/aws-cli#2639
• https://discuss.circleci.com/t/zip-does-not-support-timestamps-before-1980/13110
• Zip error on Elastic Beanstalk deploy Shippable/support#3634
• Use writestr to zip files with mtime before 1980 aws/aws-cli#2646
• NPM package has 01/01/1970 timestamps on uglify-js files (breaking build) webpack/webpack#5022

[alexlamsl]

Closing as duplicate of #2054

(I will say this one more time - if somebody can figure out why npm@5 would create tar ball containing files of what looks like UTC date of 0 that would help. Otherwise this is bound to happen again in the next npm publish.)

[fdintino]

@alexlamsl have you not read any of the discussion? This doesn't have anything to do with npm@5! npm publish doesn't change the file metadata of the local files, it uses them as it finds them in the directory where it is run. The reason it created a tar file with files created and modified at epoch 0 is because they were that way on the filesystem where it was run. I can only speculate about how they got that way on your machine.

I guarantee that if you clone the repository fresh into a different directory, bump the version, build, and publish, the files will not have UTC date of 0. REGARDLESS of which npm version you use.

[jesstelford]

Note: I have moved this reproducible case to the other issue which appears to be getting more attention.

[gabor]

maybe npm/npm#16734 ?

[alexlamsl]

maybe npm/npm#16734 ?

@gabor I think you've just hit jackpot - thanks for finding the root cause of this issue 👍

[kzc]

maybe npm/npm#16734 ?

An npm maintainer commented that it's a node 8 bug that was fixed but it's not clear what node 8 bug that is, or what release it will be in. Probably best to avoid node@8 and npm@5 for releases until this shakes out.

[not-an-aardvark]

It was apparently fixed by nodejs/node#13173, which was released as part of Node 8.1.0.

[fdintino]

Nice find @gabor! It looks like I was very much in the wrong about this not being a node bug.

@alexlamsl I hope this means there will be a re-release of 2.x packaged either with node 8.1 or node 6.x. This is not for my benefit (we resolved this issue in our code days ago, shortly after I posted in this thread, by pinning to @2.8.27 in our package.json) but for the benefit of others who have not yet been bitten by the ctime and mtime in the 2.8.28 release's tar file—that they don't have to spend time troubleshooting and diagnosing why zip or unison isn't working with a build of their project.

[jesstelford]

Excellent debugging @gabor.

@alexlamsl are you able to confirm it was that particular combo of node@>8.0.0<8.10 and Windows 7 which triggered the bug causing the 0 timestamp?

If so, can we assume a new release with a node version outside of that range would fix it?

[alexlamsl]

@jesstelford not quite Win7, but I was certainly using node/8.0.0/x64 on Windows.

So in the next backports release, I'll be using node@6 (LTS) for npm publish instead.

[STRML]

Since this release is clearly broken for some users (the last thread makes that obvious), and the root cause has been found, would you please publish2.8.29 as a repub of 2.8.28 using node@6? Republishing the same code under a new tag to fix a publish error is common (I've done it once or twice) and that will calm everyone until the next release.

[alexlamsl]

@STRML sounds like a plan - let me get out of bed first 😅