src: round nsec instead of truncating stat times

[sciolist]

Truncation of submillisecond timestamp accuracy on windows creates situations where attempting to change access or modification times becomes off by one ms. Example:

var fs = require('fs');
var targetMTime = new Date('2017-04-08T17:59:38.008Z');

fs.writeFileSync("./file1", "");
fs.utimesSync("./file1", targetMTime, targetMTime);

var actualMTime = fs.statSync("./file1").mtime;

console.log(actualMTime); // 2017-04-08T17:59:38.007Z
console.log(actualMTime.getTime() === targetMTime.getTime()); // false

This PR fixes the issue by rounding nsec rather than truncating in node_file.cc. A concern with that is that stat is a rather hot bit of code.. adding in 4 rounding operations could make a noticeable difference.

Another option could be to increment the return from toUnixTimestamp in fs.js enough to offset the precision loss, that would at least fix the issue if only node processes are updating the files.. but not particularly robust.

A third solution is to ignore the problem. It likely isn't something that affects that many users.

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• commit message follows commit guidelines

Affected core subsystem(s)

src

[mscdex]

Have you ran the existing statSync() benchmarks to compare before and after this change?

[vsemozhetbyt]

CI: https://ci.nodejs.org/job/node-test-pull-request/7608/

[vsemozhetbyt]

From a CI log:

../src/node_file.cc: In function ‘void node::FillStatsArray(double*, const uv_stat_t*)’:
../src/node_file.cc:469:121: error: ‘round’ was not declared in this scope
   X(10, atim)

[sciolist]

Mmm needs to include math for linux, had only built on Windows.
I'll run all these benchmarks, which are taking a while, and correct it.

[vsemozhetbyt]

CI: https://ci.nodejs.org/job/node-test-pull-request/7612/

[sciolist]

Linux benchmark results:

jdp@vrt:~/node$ cat c.csv | Rscript benchmark/compare.R
                                                 improvement confidence     p.value
 fs/bench-statSync.js kind="fstatSync" n=1000000     -2.21 %         ** 0.004221974
 fs/bench-statSync.js kind="lstatSync" n=1000000     -1.09 %            0.146230478
 fs/bench-statSync.js kind="statSync" n=1000000       0.67 %            0.415415495

Windows benchmark results:

C:\dev\node>Rscript benchmark\compare.R <c.csv
                                                  improvement confidence      p.value
 fs\\bench-statSync.js kind="fstatSync" n=1000000     -1.50 %         ** 1.763399e-03
 fs\\bench-statSync.js kind="lstatSync" n=1000000     -2.36 %        *** 3.457501e-06
 fs\\bench-statSync.js kind="statSync" n=1000000      -0.77 %            2.111487e-01

Pretty substantial hit. Especially for linux considering it can't even have sub-ms filetimes.
Could be ifdef'ed to only round on Windows (if that's the only supported platform with higher resolution filetimes,) but that's still not great.

[addaleax]

@refack How does uv_uptime relate to this?

[refack]

@refack How does uv_uptime relate to this?

🤦I was half asleep.

[refack]

To rectify my half-baked-sleepy-thought from before, let me suggest a different solution:
In fs.js replace the value properties with lazy getters. you get the Round and a performance gain.

 lib/fs.js | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/lib/fs.js b/lib/fs.js
index c1d8db9f8..8759dcf83 100644
--- a/lib/fs.js
+++ b/lib/fs.js
@@ -157,10 +157,14 @@ function Stats(
   this.ino = ino;
   this.size = size;
   this.blocks = blocks;
-  this.atime = new Date(atim_msec);
-  this.mtime = new Date(mtim_msec);
-  this.ctime = new Date(ctim_msec);
-  this.birthtime = new Date(birthtim_msec);
+  Object.defineProperties(fs, {
+    atime: {enumerable: true, get() {return new Date(Math.Round(atim_msec))}},
+    mtime: {enumerable: true, get() {return new Date(Math.Round(mtim_msec))}},
+    ctime: {enumerable: true, get() {return new Date(Math.Round(ctim_msec))}},
+    birthtime: {enumerable: true, get() {
+        return  new Date(Math.Round(birthtim_msec))
+    }}
+  }
 }
 fs.Stats = Stats;

[refack]

Also there's the Truncate(x + 0.5) trick that might be more performant

FYI on windows the values are only accurate to 0.1 millisecond

Time values CreationTime, LastAccessTime, LastWriteTime, and ChangeTime are expressed in absolute system time format. Absolute system time is the number of 100-nanosecond intervals since the start of the year 1601 in the Gregorian calendar.
_{from https://msdn.microsoft.com/en-us/library/windows/hardware/ff545762(v=vs.85).aspx}

[refack]

I'm not a C++ maven but maybe
(static_cast<double>((s->st_##name.tv_nsec + 500000) / 1000000))
Would be faster

Also If you're already here could you #define the magic numbers please?

[sciolist]

@refack yeah truncating and adding 0.5ms is another option. felt a bit ugly to suggest but it would fix the issue with likely lower impact on performance. rounding seems like the more correct approach, but considering the slowdown it is worth considering trunc+add.

i'm not too sure that using lazy properties on the stat object would be good.. i did consider it, but it's a bit unexpected that reading time values after stat finishes involves a function call. might not really matter, though.

[mscdex]

AFAIK the only issue with adding 0.5 and dividing would be if the value was negative, but since that should/will never be the case here, it may be safe to do. It would be interesting to see the performance difference anyway compared to just round().

[refack]

In hot spots we generally prefer performance over "the most correct solution"
So I'm +1 for the Truncate(x + 0.5) solution
But I have a feeling that the eventual new Date() is much more heavy, and at least on windows atime and birthtime are useless, so why pay for them.
So I'm +5 for the lazy properties

[refack]

but since that should/will never be the case here

name.tv_nsec is an unsigned long 👍

[mscdex]

But I have a feeling that the eventual new Date() is much more heavy, and at least on windows atime and birthtime are useless, so why pay for them.

There is/was previous discussion about these kinds of changes to fs.Stats in another thread.

[refack]

There is/was previous discussion about these kinds of changes to fs.Stats in another thread.

@mscdex couldn't find anything recent about the times... Do you remember where?

[mscdex]

@refack #12115

[refack]

@mscdex thanks!

[sciolist]

I'll try to run some more benchmarks tomorrow. Been up for a few days now, getting tired..

If we're looking into the lazy prop approach i'd expect to have to define them once on the prototype rather than in the constructor, as defineProperty is plenty more expensive than instantiating a Date. To do that we'll need a way to get the msec times outside the constructor, and also the return values should likely be cached in get(). I can give it a look when i have a little time to see what effect it would have.

[refack]

If we're looking into the lazy prop approach i'd expect to have to define them once on the prototype rather than in the constructor, as defineProperty is plenty more expensive than instantiating a Date. To do that we'll need a way to get the msec times outside the constructor, and also the return values should likely be cached in get(). I can give it a look when i have a little time to see what effect it would have.

Yes on all 👍

[refack]

The define in the ctor allowed me to hide them in a closure, but you could store the raw numbers in _XXXX members and answer #8276

[MylesBorins]

Should this land on v6.x?

[refack]

Should this land on v6.x?

Yes

• current change-set is a Windows only bug fix. It should land iff src,fs: calculate times as unsigned long #13281 lands.
• fs: expose Stats times as Numbers #13173 depends on those two.

[MylesBorins]

This is not landing cleanly, would you be willing to do a backport with the commits you mention?

[MylesBorins]

ping