Merge pull request shakacode#479 from shakacode/alleycat-at-git-alexe…

…y/replace_symlinks_copy

* Enhancements to webpack asset preparation
* Better messages when creating symlinks
* Updated documentation
* Enhanced example
* Support subdirectories with webpack assets
* Move logic for assets code to service object
* Using defaults of the env settings or else values for directories and
  regexp can be provided.

docs/additional-reading/rails-assets.md

@@ -1,19 +1,83 @@
-## Rails assets
+# Rails assets and the Extract Text Plugin
 
-### Problem
-When client js uses images in render methods, e.g. `<img src='...' />` or in css, e.g. `background-image: url(...)` 
-these assets fail to load. This happens because rails adds digest hashes to filenames 
-when compiling assets, e.g. `img1.jpg` becomes `img1-dbu097452jf2v2.jpg`. 
+The [Webpack file loader](https://github.com/webpack/file-loader) copies referenced files to
+the destination output directory, with an MD5 hash. The other term for this is a "digest".
+
+> By default the filename of the resulting file is the MD5 hash of the file's contents with
+the original extension of the required resource.
+
+The most common use cases for Webpack processed files are images used for backgrounds in
+CSS and fonts for CSS. However, this applies to any file that might be processed using the
+Webpack file loader.
+
+## The Problem
+To understand the problem, it helps to read this article: 
+[What is fingerprinting and why should I care](http://guides.rubyonrails.org/asset_pipeline.html#what-is-fingerprinting-and-why-should-i-care-questionmark)
+Basically, when Rails prepares assets for production deployments, it also adds a digest
+to the file names. E.g., `img1.jpg` becomes `img1-dbu097452jf2v2.jpg`. 
 
 When compiling its native css Rails transforms all urls and links to digested 
 versions, i.e. `background-image: image-url(img1.jpg)` becomes 
 `background-image: url(img1-dbu097452jf2v2.jpg)`. However this doesn't happen for js and 
 css files compiled by webpack on the client side, because they don't use 
-`image-url` and `asset-url` and therefore assets fail to load.
+`image-url` and `asset-url`. Without some fix, these assets would fail to load.
 
-### Solution
+When Webpack's client JavaScript uses images in render methods, e.g. `<img src='...' />` or 
+in css, e.g. `background-image: url(...)` The code (such as the CSS) generated by the Webpack
+will have the Webpack digested name (MD5 hash). Since the Webpack generated CSS expects
+just one level of "digesting", this "double-digesting" from Rails will cause such these assets
+fail to load. 
 
-React on Rails creates symlinks of non-digested versions to digested versions when doing a Rails assets compile.
-The solution is implemented using `assets:precompile` after-hook. The assets for symlinking
-are defined by `config.symlink_non_digested_assets_regex` in `config/initializers/react_on_rails.rb`.
+## The Solution: Symlink Original File Names to New File Names
+React on Rails creates symlinks of non-digested versions (original webpack digested file names) 
+to the Rails deployed digested versions when doing a Rails assets compile. The solution is 
+implemented using `assets:precompile` after-hook in 
+file [lib/tasks/assets.rake](../../lib/tasks/assets.rake)
+The assets for symlinking are defined by `config.symlink_non_digested_assets_regex` in 
+`config/initializers/react_on_rails.rb`.
+
+## Disabling the Symlinking
 To disable symlinks set this parameter to `nil`.
+
+
+## Example from /spec/dummy
+
+If you run
+
+```
+cd spec/dummy
+RAILS_ENV=production bundle exec rake assets:precompile
+rails s -e production
+```
+
+You will see this. This shows how the file names output by rails. Note the original names after
+being processed by Webpack are just MD5's.
+
+```
+I, [2016-07-17T23:46:56.301981 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/server-bundle-42935dea382802a27e91b7df444a2813f74b4e6a0fce5606d863aaa10c0623d7.js
+I, [2016-07-17T23:46:56.305649 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/server-bundle-42935dea382802a27e91b7df444a2813f74b4e6a0fce5606d863aaa10c0623d7.js.gz
+I, [2016-07-17T23:46:56.370390 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/application_static-dfa728160c3cdebc633c2f6fb3823411530b307044f4dfe460790eef00b4e421.js
+I, [2016-07-17T23:46:56.370566 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/application_static-dfa728160c3cdebc633c2f6fb3823411530b307044f4dfe460790eef00b4e421.js.gz
+I, [2016-07-17T23:46:56.372895 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/application_static-17ed778d5061d4797556632b7bfbf405e067d9e7f140060a7f56a09788251f16.css
+I, [2016-07-17T23:46:56.373012 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/application_static-17ed778d5061d4797556632b7bfbf405e067d9e7f140060a7f56a09788251f16.css.gz
+I, [2016-07-17T23:46:56.374531 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/2ac2dd94f9b7e54292f6d051f1e4e756-ab14eebb171a9a5c9bfdeb2f88933d2dc4904ea8bb09444984e52b13d230e251.svg
+I, [2016-07-17T23:46:56.374818 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/2ac2dd94f9b7e54292f6d051f1e4e756-ab14eebb171a9a5c9bfdeb2f88933d2dc4904ea8bb09444984e52b13d230e251.svg.gz
+I, [2016-07-17T23:46:56.392207 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/5cf5db49df178f9357603f945752a1ef-033650e1d6193b70d59bb60e773f47b6d9aefdd56abc7ccdba3c7bed4e57ccad.png
+I, [2016-07-17T23:46:56.393208 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/8970f5e1e92aea933b502a2d73976b76-877bde3739dc7080c3fb00ee9012db6f21ed0dbbf11cd596dbb6e1a35bfb71f9.png
+I, [2016-07-17T23:46:56.395490 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/ecb4572a5e478b107dfcb60c16a7eefa-6d1ab3741d5a164dc2aab48bb74429aebe2e2e29606feca581081697624dc18c.ttf
+I, [2016-07-17T23:46:56.395846 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/ecb4572a5e478b107dfcb60c16a7eefa-6d1ab3741d5a164dc2aab48bb74429aebe2e2e29606feca581081697624dc18c.ttf.gz
+I, [2016-07-17T23:46:56.396979 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fbd0d00cc9b670f05c17893a40da08d0-5731789fd0d7847a582b52b55a83e7a0ad4684acd5a9b487557635a08c112d0e.svg
+I, [2016-07-17T23:46:56.397669 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fbd0d00cc9b670f05c17893a40da08d0-5731789fd0d7847a582b52b55a83e7a0ad4684acd5a9b487557635a08c112d0e.svg.gz
+I, [2016-07-17T23:46:56.399261 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fc2dcaaf2057331ff76c5d37e1aa7056-efba50c701b697fc8160603b9e876adcf47511f35af68701db285272c965a45f.svg
+I, [2016-07-17T23:46:56.399660 #77382]  INFO -- : Writing /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fc2dcaaf2057331ff76c5d37e1aa7056-efba50c701b697fc8160603b9e876adcf47511f35af68701db285272c965a45f.svg.gz
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/2ac2dd94f9b7e54292f6d051f1e4e756-ab14eebb171a9a5c9bfdeb2f88933d2dc4904ea8bb09444984e52b13d230e251.svg to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/2ac2dd94f9b7e54292f6d051f1e4e756.svg
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/2ac2dd94f9b7e54292f6d051f1e4e756-ab14eebb171a9a5c9bfdeb2f88933d2dc4904ea8bb09444984e52b13d230e251.svg.gz to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/2ac2dd94f9b7e54292f6d051f1e4e756.svg.gz
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/5cf5db49df178f9357603f945752a1ef-033650e1d6193b70d59bb60e773f47b6d9aefdd56abc7ccdba3c7bed4e57ccad.png to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/5cf5db49df178f9357603f945752a1ef.png
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/8970f5e1e92aea933b502a2d73976b76-877bde3739dc7080c3fb00ee9012db6f21ed0dbbf11cd596dbb6e1a35bfb71f9.png to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/8970f5e1e92aea933b502a2d73976b76.png
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/ecb4572a5e478b107dfcb60c16a7eefa-6d1ab3741d5a164dc2aab48bb74429aebe2e2e29606feca581081697624dc18c.ttf to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/ecb4572a5e478b107dfcb60c16a7eefa.ttf
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/ecb4572a5e478b107dfcb60c16a7eefa-6d1ab3741d5a164dc2aab48bb74429aebe2e2e29606feca581081697624dc18c.ttf.gz to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/ecb4572a5e478b107dfcb60c16a7eefa.ttf.gz
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fbd0d00cc9b670f05c17893a40da08d0-5731789fd0d7847a582b52b55a83e7a0ad4684acd5a9b487557635a08c112d0e.svg to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fbd0d00cc9b670f05c17893a40da08d0.svg
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fbd0d00cc9b670f05c17893a40da08d0-5731789fd0d7847a582b52b55a83e7a0ad4684acd5a9b487557635a08c112d0e.svg.gz to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fbd0d00cc9b670f05c17893a40da08d0.svg.gz
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fc2dcaaf2057331ff76c5d37e1aa7056-efba50c701b697fc8160603b9e876adcf47511f35af68701db285272c965a45f.svg to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fc2dcaaf2057331ff76c5d37e1aa7056.svg
+React On Rails: Symlinking /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fc2dcaaf2057331ff76c5d37e1aa7056-efba50c701b697fc8160603b9e876adcf47511f35af68701db285272c965a45f.svg.gz to /Users/justin/shakacode/react_on_rails/spec/dummy/public/assets/fc2dcaaf2057331ff76c5d37e1aa7056.svg
+```

lib/generators/react_on_rails/templates/base/base/config/initializers/react_on_rails.rb.tt

@@ -70,8 +70,8 @@ ReactOnRails.configure do |config|
   config.server_render_method = "ExecJS"
 
   # Client js uses assets not digested by rails.
-  # For any asset matching this regex, non-digested symlink will be created
+  # For any asset matching this regex, non-digested symlink will be created (what webpack's css wants)
   # To disable symlinks set this parameter to nil.
-  config.symlink_non_digested_assets_regex = /\.(png|jpg|jpeg|gif|tiff|woff|ttf|eot|svg)/
+  config.symlink_non_digested_assets_regex = /\.(png|jpg|jpeg|gif|tiff|woff|ttf|eot|svg|map)/
 
 end

lib/react_on_rails/assets_precompile.rb

@@ -0,0 +1,110 @@
+module ReactOnRails
+  class AssetsPrecompile
+    # Used by the rake task
+    def default_asset_path
+      dir = File.join(Rails.configuration.paths["public"].first,
+                      Rails.configuration.assets.prefix)
+      Pathname.new(dir)
+    end
+
+    def initialize(assets_path: nil,
+                   symlink_non_digested_assets_regex: nil,
+                   generated_assets_dir: nil)
+      @assets_path = assets_path.presence || default_asset_path
+      @symlink_non_digested_assets_regex = symlink_non_digested_assets_regex.presence ||
+                                           ReactOnRails.configuration.symlink_non_digested_assets_regex
+      @generated_assets_dir = generated_assets_dir.presence || ReactOnRails.configuration.generated_assets_dir
+    end
+
+    # target and symlink are relative to the assets directory
+    def symlink_file(target, symlink)
+      target_path = @assets_path.join(target)
+      symlink_path = @assets_path.join(symlink)
+      target_exists = File.exist?(target_path)
+
+      # File.exist?(symlink_path) will check the file the sym is pointing to is existing
+      # File.lstat(symlink_path).symlink? confirms that this is a symlink
+      symlink_already_there_and_valid = File.exist?(symlink_path) &&
+                                        File.lstat(symlink_path).symlink?
+      if symlink_already_there_and_valid
+        puts "React On Rails: Digested #{symlink} already exists indicating #{target} did not change."
+      elsif target_exists
+        if File.exist?(symlink_path) && File.lstat(symlink_path).symlink?
+          puts "React On Rails: Removing invalid symlink #{symlink_path}"
+          `cd #{@assets_path} && rm #{symlink}`
+        end
+        # Might be like:
+        # "images/5cf5db49df178f9357603f945752a1ef.png":
+        # "images/5cf5db49df178f9357603f945752a1ef-033650e1d6193b70d59bb60e773f47b6d9aefdd56abc7cc.png"
+        # need to cd to directory and then symlink
+        target_sub_path, _divider, target_filename = target.rpartition("/")
+        _symlink_sub_path, _divider, symlink_filename = symlink.rpartition("/")
+        puts "React On Rails: Symlinking #{target} to #{symlink}"
+        dest_path = File.join(@assets_path, target_sub_path)
+        `cd #{dest_path} && ln -s #{target_filename} #{symlink_filename}`
+      end
+    end
+
+    def symlink_non_digested_assets
+      # digest ==> means that the file has a unique sha so the browser will load a new copy.
+      # Webpack's CSS extract-text-plugin copies digested asset files over to directory where we put
+      # we deploy the webpack compiled JS file. Since Rails will deploy the image files in this
+      # directory with a digest, then the files are essentially "double-digested" and the CSS
+      # references from webpack's CSS would be invalid. The fix is to symlink the double-digested
+      # file back to the original digested name, and make a similar symlink for the gz version.
+      if @symlink_non_digested_assets_regex
+        manifest_glob = Dir.glob(@assets_path.join(".sprockets-manifest-*.json")) +
+                        Dir.glob(@assets_path.join("manifest-*.json"))
+        if manifest_glob.empty?
+          puts "Warning: React On Rails: expected to find .sprockets-manifest-*.json or manifest-*.json "\
+                   "at #{@assets_path}, but found none. Canceling symlinking tasks."
+          return -1
+        end
+        manifest_path = manifest_glob.first
+        manifest_data = JSON.load(File.new(manifest_path))
+
+        # We realize that we're copying other Rails assets that match the regexp, but this just
+        # means that we'd be exposing the original, undigested names.
+        manifest_data["assets"].each do |original_filename, rails_digested_filename|
+          # TODO: we should remove any original_filename that is NOT in the webpack deploy folder.
+          next unless original_filename =~ @symlink_non_digested_assets_regex
+          # We're symlinking from the digested filename back to the original filename which has
+          # already been symlinked by Webpack
+          symlink_file(rails_digested_filename, original_filename)
+
+          # We want the gz ones as well
+          symlink_file("#{rails_digested_filename}.gz", "#{original_filename}.gz")
+        end
+      end
+    end
+
+    def delete_broken_symlinks
+      Dir.glob(@assets_path.join("*")).each do |filename|
+        next unless File.lstat(filename).symlink?
+        begin
+          target = File.readlink(filename)
+        rescue
+          puts "React on Rails: Warning: your platform doesn't support File::readlink method." /
+               "Skipping broken link check."
+          break
+        end
+        path = Pathname.new(File.dirname(filename))
+        target_path = path.join(target)
+        unless File.exist?(target_path)
+          puts "React on Rails: Deleting broken link: #{filename}"
+          File.delete(filename)
+        end
+      end
+    end
+
+    def clobber
+      dir = Rails.root.join(@generated_assets_dir)
+      if dir.present? && File.directory?(dir)
+        puts "Deleting files in directory #{dir}"
+        FileUtils.rm_r(Dir.glob(Rails.root.join("#{@generated_assets_dir}/*")))
+      else
+        puts "Could not find generated_assets_dir #{dir} defined in react_on_rails initializer: "
+      end
+    end
+  end
+end

lib/react_on_rails/configuration.rb

@@ -57,7 +57,7 @@ def self.configuration
       webpack_generated_files: [],
       rendering_extension: nil,
       server_render_method: "",
-      symlink_non_digested_assets_regex: /\.(png|jpg|jpeg|gif|tiff|woff|ttf|eot|svg)/,
+      symlink_non_digested_assets_regex: /\.(png|jpg|jpeg|gif|tiff|woff|ttf|eot|svg|map)/,
       npm_build_test_command: "",
       npm_build_production_command: ""
     )