createProgram: provide file content with file objects independent of fs access - I. e. for Meteor Build System

[D1no]

To enable meteor developers the use of TypeScript in meteor packages and application code, we are currently working on a compiler plugin that needs to integrate into the meteor specific build system.
_note: the 9th most stared Github repo, before ruby/rails, is the meteor js framework

The problem is that the meteor build system instruments a caching and bundling system that obfuscates access to the file system, which seems to challenge the fs and typescript api coupling.

I. e. during meteor build an [1]array of input file objects that each incl. filename, -path, -hash and -content (data) properties is passed to the compile plugin (here typescript) and expected to be returned as [2]transpiled content via a method handler. Without touching the file-system.

Analog to the "A minimal compiler" API example, it seems to be easy to cover [2] by providing a custom compileHandler: ts.WriteFileCallback into program.emit(undefined, compileHandler) to pass the transpiled content back upstream.

export function compile(fileNames: string[], options: ts.CompilerOptions): void {
    var program = ts.createProgram(fileNames, options);
    var emitResult = program.emit();
//...

But can [1] lead to a full Program instance without providing fs-access just with "virtual" file paths and file contents?

• It would be convenient to have ts.createProgram accept file objects that encapsulate file properties such as content etc. to make the compiler file-system agnostic.

Any given advice to prevent redundant disk access while maintaining source maps, incremental builds and diagnostics inside this build chain?

[vladima]

createProgram accepts optional compilerHost argument. This host is responsible for all IO related operations and by default is is file system based. You can always provide alternative implementation that uses some in-memory cache to provide answers.

[weswigham]

Simply replacing the host when instantiating a service should usually suffice, but failing that, assigning ts.sys a host-specific implementation of a System should cover everything.

[vladima]

I would consider replacement of sys to be more like a hack than a recommended solution. The fact the default version of compilerHost uses sys is an implementation detail and I would not recommend to rely on it

[D1no]

Thanks @vladima! compilerHost looks like what we need, though not sure how to instrument it yet. Are there by now better API documentations except for the directives?

[jbrantly]

If all you need is transpilation without diagnostics on inter-file dependencies, check out the transpile functionality.

If you need more (you mentioned incremental builds and diagnostics) then I think you should use the language service instead. I used the language service to build something like what you're talking about for webpack, and while it has a lot of webpack-specific stuff it may help guide you. In particular take a look at the getScriptSnapshot implementation which answers your initial question about loading from memory.

[D1no]

Exactly, a simple transpile would defeat the purpose of compile assurance :)
I originally hoped to get around the language service implementation (after all, its not an IDE) but the getScriptSnapshot looks like a plan in getting around a parallel build implementation.

Since the meteor build system plays the role of the file watcher, starting and killing the process at will, I'd need to check how to translate that into some cache layer.

[mhegazy]

looks like the original issue has been handled. please reopen if this is not the case.

[ChrisProlls]

Hi @D1no, I have the same needs as yours.
After reading some docs about the LanguageService, I see that I need to implements methods, like getCurrentDirectory, readDirectory or fileExists, but I already have the file content in memory.
Any ideas ?