Usage report - concatjs

[achew22]

I just finished getting ConcatJS set up in my project and while the memories are fresh I wanted to document the things that left me slightly puzzled. I think I'm probably the first person to do this, so it might give you some direction on how the integration might be improved.

1. Thanks. I'm really happy that this exists. Writing my own concatjs would have been a PITA. Looking through the code it seems like there are some tricky edge cases that would have tripped me up. Thanks!

2. Visibility is a problem. You're in an interesting place where it doesn't make sense for someone using the default go toolchain to import concatjs. What would they do with it? As such the Bazel visibility is the one that really matters. Unfortunately it is currently set to //internal:__subpackages__. This is almost a showstopper except for the fact that you can add build --nocheck_visibility to your projects bazel.rc. This seems like the most important thing to fix.

3. Target naming. I think the rules_go community is starting to settle around using gazelle to manage things. As such the target name of @build_bazel_rules_typescript//internal/concatjs is not what is expected by Gazelle. It expects @build_bazel_rules_typescript//internal/concatjs:go_default_library. Further there is a mismatch between the workspace name build_bazel_rules_typescript and the importpath definition of github.com/bazelbuild/rules_typescript/internal/concatjs/concatjs. For gazelle to import that correctly, it would need to have importpath set to importpath = "bazel.build/rules_typescript/internal/concatjs/concatjs". Also, the duplication of concatjs on the import path at the end appears to be an artifact of the Google3 implementation of rules_go that was not carried forward during the open source rewrite. Interestingly, I think this is a thing the Gazelle team is aware of and would like to address. They have a tracking bug Index build files in external repositories bazel-gazelle#12 to make gazelle work properly with this code, but I still think it would be better to run Gazelle on the code during the export process to align it with the OSS community's expectations better.

4. Content security policies. concatjs uses eval to load the scripts into the page. That's a totally fine thing to do, but it did interfere with the CSP I already had on the page. It might be worth noting that CSPs are essentially incompatible with this approach of doing the devserver and that you will have to add a flag to disable CSPs as necessary.

5. Serving paths. When I initially plugged in the concatjs.ServeConcatenatedJS handler, I put it at /js/concat.js. Earlier in the file I had registered a router.PathPrefix("/js/").Handler(...). Turns out you can't graft into the hierarchy over a PathPrefix. Probably not a concern for most people, but it might be worth noting that in the docs.

6. preScripts. I copied the style of having a list of preScripts that were passed into concatjs rather than including scripts in the page. I needed to manually include RequireJS, which I did by adding a data dependency on @build_bazel_rules_typescript_devserver_deps//:node_modules/requirejs/require.js and with the following block of go, before my call to concatjs.ServeConcatenatedJS:

   	{ // Pull in RequireJS
   		buf, err := ioutil.ReadFile("external/build_bazel_rules_typescript_devserver_deps/node_modules/requirejs/require.js")
   		if err != nil {
   			panic(err)
   		}
   
   		preScripts = append(preScripts, string(buf))
   	}
   

7. postScripts. Since I used the style of preScripts, it only seems reasonable that I would use the style of postScripts. I needed to add postScripts = append(postScripts, require(["my/entry/point"]);) before the call to the handler as well.

8. The correct base dir. It was a little tricky figuring out the correct working dir inside my server.

   import (                                                                                                                                                                            
   ...
     "os"
     "path"
   ...
   )
   ...
   	wd, _ := os.Getwd()
   	// wd is the path $RUNFILES/$WORKSPACE_NAME.
   	// concatjs expects to be in $RUNFILES so cd .. to get to there.
   	rootRunfiles := path.Dir(wd)
   	router.Handle("/debug/concat.js",
   		concatjs.ServeConcatenatedJS(
   			"$WORKSPACE_NAME/path/to/js/devserver.MF",
   			rootRunfiles, preScripts, postScripts, nil))
   

9. tags. I also needed to add the following tags to the target in order to have iBazel do "the right thing(TM)".

   tags = [
       # Tell ibazel not to restart the devserver when its data changes.
       "ibazel_notify_changes",
       # Tell ibazel to serve the live reload script, since we expect a
       # browser will connect to this program.
       "ibazel_live_reload",
   ],
   

10. Depending on the right data. When it came time to depend on the correct resources, I was left a little befuddled. Initially I thought I would only need to depend on the ts_library (but that doesn't generate a .MF file). Then I thought that I would only need to depend on the ts_devserver and that it would provide, transitively, access to the compiled .js files. That was not the case. I needed to depend explicitly on both.

    DEVELOPMENT_DATA = [
        "@build_bazel_rules_typescript_devserver_deps//:node_modules/requirejs/require.js",                                                                                             
        "//path/to/js:my_lib", # ts_library(name = "my_lib")
        "//path/to/js:devserver.MF", # Generated by ts_devserver(name = "devserver")
        "//path/to/js:devserver", # Same as above
    ]
    

Sorry to wall of text, but as I said I wanted to write down all the little pains that you can only see the first time you do an integration.

All and all I would say, GREAT SUCCESS! 🎉. ConcatJS seems to be working great in my project and I'm really enjoying the diminished compile times and iteration speed. Kudos!

[alexeagle]

Thanks for keeping such great notes! What is the next step here? Seems like we need to extract some of this into docs, and file issues for the rest?
Note there's a general issue for documenting how to bring your own devserver - I plan to write a Java example one soon. Some of the docs here are go-specific and some would apply there as well (like the ibazel tags).
If you have time to contribute some of the docs that would be greatly appreciated, of course!

[alexeagle]

I'm not sure what to do, @achew22 do you think we should just transcribe your notes to the wiki or something? I don't want to lose them.

[achew22]

What would your thoughts be on moving it into the concatjs directory as a checked in .md file?

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 90 days. It will be closed if no further activity occurs in two weeks. Collaborators can add a "cleanup" or "need: discussion" label to keep it open indefinitely. Thanks for your contributions to rules_nodejs!

[gregmagolan]

No longer in scope for rules_nodejs