Replace colons in transform cache filenames #8353
Conversation
etr2460
force-pushed
the
fix-cache-file-colons
branch
from
92b06d9
to
051c575
Compare
Codecov Report
@@ Coverage Diff @@
## master #8353 +/- ##
==========================================
+ Coverage 62.33% 62.34% +<.01%
==========================================
Files 266 266
Lines 10735 10736 +1
Branches 2610 2610
==========================================
+ Hits 6692 6693 +1
Misses 3460 3460
Partials 583 583
Continue to review full report at Codecov.
|
The hash does contain the rest of the filename too, but technically hashes for the files |
|
We should hash the original filename, shouldn't we? |
|
Ah, I see what you mean. Right now we aren't hashing the filename, only the file contents. I'll make the update to hash the filename as well |
| @@ -94,6 +94,7 @@ export default class ScriptTransformer { | |||
| rootDir: this._config.rootDir, | |||
| }), | |||
| ) | |||
| .update(filename) | |||
There was a problem hiding this comment.
This isn't necessary because getCacheKey includes the filename already.
There was a problem hiding this comment.
Yes I thought so as well, my comment was only to seek confirmation ^^
| @@ -121,11 +123,11 @@ export default class ScriptTransformer { | |||
| // Create sub folders based on the cacheKey to avoid creating one | |||
| // directory with many files. | |||
| const cacheDir = path.join(baseCacheDir, cacheKey[0] + cacheKey[1]); | |||
| const cacheFilenamePrefix = path | |||
| .basename(filename, path.extname(filename)) | |||
| .replace(/:/g, '_COLON_'); | |||
There was a problem hiding this comment.
Is colon really the only character that might cause issues here? Especially when you consider it's possible for a cache to be created on one system and then be used on another (this is what we do at Facebook and I'm sure it's not totally unique).
Stepping back a second, the extension being added to the file seems odd to me in the first place. With your .update(filename) added above, I don't see a reason we can't just drop this entire find/replace and then also stop saving the file with the extname prefix. The cache key has the whole filename in it. That'll do.
@SimenB let me know if you have any thoughts but here is my recommended course of action:
- Remove
.update(filename)from the custom cache key code path since it has filename available already. - Keep
.update(filename)in the built-in cache key code path. - Remove this find/replace and stop using the
extnameat all.
That should resolve the core issue and other similar issues we don't know about yet.
There was a problem hiding this comment.
There is obviously no technical necessity to have the prefix because it's already in the hash (and stripping away most of the path would make it useless for collision avoidance anyway), but I would guess it's there to make debugging / working on cache related things easier?
There was a problem hiding this comment.
Yeah, good point. I'm still in favor of killing it if it's causing issues.
There was a problem hiding this comment.
If the filename prefix is helpful with debugging cache related things, how about we prefix with just the alphanumeric characters from the filename? That should solve any issues across systems but retain the human friendly context
There was a problem hiding this comment.
@scotthovestadt you fine with this too? I think it's a good way to handle it
etr2460
force-pushed
the
fix-cache-file-colons
branch
from
6902c63
to
0737f6b
Compare
|
@scotthovestadt any other changes that you'd like me to make here? |
|
@et2460 if you rebase on master then we're good to merge :) |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Summary
Fixes #6306 by replacing colons in filenames with the symbol
_COLON_Test plan
Run
yarn jest script_transformer.test.js. Let me know if any further testing is required!