I believe that changing any file in the source causes the TS flake to redownload all dependencies.
A fix would be to probably create the node_modules by only copying the package.json and package-lock.json in hopes of convincing nix that the dependencies haven't changed.