support for plugins / support for jsx

[davidreher]

Hey,

I wanted to use jsx in typescript. Therefore I planned to use something like ts-jsx-loader which replaces the jsx with plain js before passing it to the typescript compiler. At the moment I do not see any other possibility but to fork and adjust tsify ...

Do you have any other suggestions?

[smrq]

As a React neophyte myself, I have a personal interest in making this work. However, I don't want to build in any React-specific extensions into tsify, and I don't think plugins for plugins is a good pattern either. The right way to do this would be to create a Browserify transform to preprocess the input before passing it off to tsify. However, this has some complications. I will try to explain; note that while my knowledge of the inner workings of tsify is basically complete, my understanding of Browserify's internals is just based on my experience in writing tsify and is thus potentially flawed.

How it works today

tsify currently makes the assumption that it is the first transformation step in the pipeline. The reason for this is somewhat complex, and is related to both TypeScript's and Browserify's module resolution steps. The TypeScript compiler needs to do its own module resolution in order to compile efficiently and perform complete type checking. If module A requires in module B, then it needs to compile module B first to get its typing information in order to determine whether module A is correctly typed. Thus, a request to compile A.ts will generate output for both A.js and B.js. This means that some internals in the compiler request B.ts from the file system. (This uses TypeScript's module loading semantics for searching for that file, which actually differ slightly from the Node module loading semantics; I don't think that matters here, but it's good to know.)

The code that performs the actual filesystem work to turn a filename into file contents lives here.

Note that, as it stands now, this means that the TypeScript compiler can request files that, up to that point, were not even necessarily known to be part of the source. They don't have to be set up within Browserify at all. That means that they aren't part of the Browserify pipeline. This is the reason that tsify currently assumes that it is the beginning of the transformation pipeline -- because it can pull things into the pipeline without sending them through all of the previous stuff.

That's the first half of how tsify works. The second half is trivial by comparison, but builds upon that assumption that it is the first transformation in the pipeline. tsify registers a Browserify transform which is quite simple: it transforms a file into the cached compiled output from the compilation that we just did. It does this by throwing away the stream contents entirely and just looking at the filename associated with the stream contents, since the compilation has already been completed. At first the transform is only called on files passed explicitly into Browserify by your build setup.

Then, Browserify parses the transformed output for require() calls, and pulls additional files into the pipeline to be transformed (using Node module resolution semantics; again, I don't think that the difference matters too much here). Of course, we know that those files have already been compiled by the TypeScript compilation step earlier, because each require() was already resolved by the compiler in order to perform type checking. So each additional file that Browserify pulls in should already be in the compilation cache, and our transform should work just fine.

In the unlikely event that Browserify attempts to transform a TypeScript file not in the compilation cache, we do an extra compilation step immediately, and suffer a performance penalty as as result. (As far as I know, this can only happen if you require() a TypeScript file from a plain JavaScript file.)

How to fix?

So, uh, this will definitely be complicated. There's sort of a weird circular problem: if you have A.ts depend on B.ts, then the TypeScript compiler needs the contents of B.ts in order to generate output for A.ts. However, Browserify needs the output of the entire transformation pipeline for A.js in order to determine that it needs to put B.ts into the pipeline to be transformed. Any fix would have to resolve this circularity somehow. I have some half-formed ideas, but definitely nothing concrete and guaranteed to work.

I might get to this sooner rather than later, but I do have quite a lot on my plate at the moment. Discussion (or even PRs!) super welcome.

[davidreher]

Hm, as far as I understand there is even the bigger problem that I cannot run any transformation steps before tsify has started, only after tsify compilation is complete ...

[smrq]

Yeah, that's basically for two reasons. One, because tsify inserts its transform at the beginning of the list of other transforms. This is trivial to change, but it does that for the reason of... Two, because it hits the filesystem directly instead of pulling file contents from the pipeline, because of the circular issue.

[smrq]

Note that it is possible that this will be pulled into the core compiler at some point: microsoft/TypeScript#2673

[smrq]

More relevant discussion: microsoft/TypeScript#3203

[tejacques]

JSX support was added to Typescript master recently -- I'm guessing it just needs to be released to npm before you can update the version of tsc that tsify uses?

[smrq]

Yes, it's already available in the tsc nightly. (Woot!) The nightly is available on npm as ntypescript, but #47 needs to happen to pull it in before it gets pushed as a real version.

[3cp]

tsify inserts its transform at the beginning of the list of other transforms.

@smrq I hit this same "issue" while trying to implement Aurelia 2 conventions for browserify. We need to inject some code (so user don't need to write boilerplate) to the source code before compiler sees it. I can successfully do it with babelify, but only find out tsify bypassed my transform. I am not sure what I can do to support TS+browserify.

[cartant]

We need to inject some code (so user don't need to write boilerplate) to the source code before compiler sees it.

As outlined above, this is not possible with tsify's implementation:

tsify currently makes the assumption that it is the first transformation step in the pipeline.

Perhaps you could use a TypeScript transform to do what you need? IDK, but reimplementing tsify to support prior transformations is something that is just not going to happen.

[3cp]

Thx @cartant for the information! We will hold TS+browserify support now. BTW TypeScript transform requires direct compiler API call which tsify needs to open that up. I am not asking for the feature, as it also requires significant investment from our side too.