Multiple UMD typings in one file.d.ts

[mihailik]

TypeScript Version: 2.4.1

[1] Simplistic scenario: I am Facebook and I want to ship typings for both @types/react and @types/react-dom in one file.

Currently those are 2 files, and they use UMD typings with export as namespace. You can:

• either move them to be non-script module-only with multiple declare module "react", declare module "react-dom";
• or make them both scripts with declare namespace React, declare namespace ReactDOM inside.

---

[2] Another more realistic scenario: plugins/extensions.

I am GitHub and I want to enable extensions to my shiny new GitHub Desktop (Electron) app. I want to ship a bundled EverythingYouNeed.d.ts file, embedding typings for the specific React version, for the specific RX version, and of course GitHub-specific APIs and extension points' typings.

At runtime I load React, ReactDOM, RX and bunch of GitHub APIs on global — so technically I can manually edit 3rd party typings from UMD style to namespaced style, before bundling them together. Extensions will just take everything off global. But it's a lot of fiddly work over external assets (DTS). Maintenance cost is high.

---

[3] My actual realistic scenario is similar to GitHub one above, but in a corporate environment. I am preparing a tooling for the individual departments to use as a base for creating HTML apps. I want it to be as bundled and blackboxed as possible. Specific 3rd party libraries, specific versions etc.

As much as I can bundle JS, HTML and CSS and even compress, currently I am forced either to distribute a large number of individual decl.d.ts, or hand-edit those to allow bundling together.

For the interest of disclosure, currently I partially process external DTS files with RegExes, and partially manually edit to fit them in the bundle. Quite embarrassing!

Suggestion

Two things:

• Export specific module as a specific namespace
• Module-aware syntax in non-module files

Exporting specific module as namespace

react-bundled.d.ts

declare module "react" {
 // declarations for React stuff...
}
export module "react" as namespace React;  <-- see export specific module

declare module "react-dom" {
 // declarations for ReactDOM stuff...
}
export module "react-dom" as namespace ReactDOM;  <-- see export specific module

Module-aware syntax in scripts

react-bundle-maker.ts

declare import * as React from "react";
declare import * as ReactDOM from "react-dom";

I tell the compiler: consider typings for module X imported by the external forces. Now let me just use it.

And if I run that react-bundle-maker.ts file from above through tsc --declaration, I expect to get the same output as react-bundled.d.ts above.

Why these suggestions

These two suggestions (with possible tweaks etc.) avoid dilemmas and problems of various kinds of loaders and bundlers as discussed in 4433 Bundling TS module type definitions, 6319 Support 'target':'es5' with 'module':'es6' and 4434 Bundling TS modules.

External loaders are dealing OK with runtime loading, but TS needs to improve its story on compile-time with modules — particularly in case of bundling for a large app scenario.

One specific hole I didn't want to dig is specifying the 'root' module for bundling. Ability to use declare import in scripts solves that neatly.

If we allow features I suggested, or something other to that effect, we'll have much smoother bundling workflow for DTS files.

[mihailik]

@weswigham you've filed the original #4433 and #4434 and done several crucial commits in this area. Can you please comment?

@mhegazy you've commented on and filed issues in this area, can you please comment?

Many thanks, and hope this makes this difficult area slightly easier to deal with!

[weswigham]

@mihailik - bundling was for packaging a module's worth of DTS files into one file, not for packing multiple modules into one. There wasn't really even a compelling reason to package multiple DTS files into one other than "people prefer to copy one file over 10", which really doesn't matter when its distributed via a package manager anyway. (Which is part if why we never merged it - the other reason being the other motivating reason was rewriting as ambient modules, which we removed the need for with umd declarations)

Packaging multiple modules into a single DTS file just seems... Pointless? It's adding coupling for coupling's sake and adding complexity to the compiler to boot? There's no reason not to just ship them as two separate files... They do refer to two separate modules after all.

Long story short, bundling provides no benefits over just shipping a collection of multiple .d.ts files, in any scenario, since its only the TS compiler that consumes them, and it would rather see many individual files, since it simplifies module resolution rules for most users and shows clear boundaries for checking.

Do you have any actual use cases which are not solved by other features? What reason do you have to want to cram the generated types for two libraries into one file? Your maintenance costs seem significantly higher specifically because you're trying to cram everything into one file.

Let me call out better ways to handle your other use-cases:

I am GitHub and I want to enable extensions to my shiny new GitHub Desktop (Electron) app. I want to ship a bundled EverythingYouNeed.d.ts file, embedding typings for the specific React version, for the specific RX version, and of course GitHub-specific APIs and extension points' typings.

At runtime I load React, ReactDOM, RX and bunch of GitHub APIs on global — so technically I can manually edit 3rd party typings from UMD style to namespaced style, before bundling them together. Extensions will just take everything off global. But it's a lot of fiddly work over external assets (DTS). Maintenance cost is high.

Ship a package.json (or init tool that makes one) instead. Don't use a .d.ts for versioning. It's not for that. The package manager does it excellently already, and the compiler understands it. Just make it list the types you care about as dev dependencies. (You may also ship a default tsconfig which lists the global types you'd like imported from those downloaded, too, if you with to save your users from writing triple slash lib references themselves.)

My actual realistic scenario is similar to GitHub one above, but in a corporate environment. I am preparing a tooling for the individual departments to use as a base for creating HTML apps. I want it to be as bundled and blackboxed as possible. Specific 3rd party libraries, specific versions etc.

As much as I can bundle JS, HTML and CSS and even compress, currently I am forced either to distribute a large number of individual decl.d.ts, or hand-edit those to allow bundling together.

For the interest of disclosure, currently I partially process external DTS files with RegExes, and partially manually edit to fit them in the bundle. Quite embarrassing!

Bundling and minifying js, HTML, and CSS is done for performance reasons. There is no reason to do the same for TS, and unbundled TS still represents the same API as the bundled version of a file (barring major transforms TS doesn't know about). If you're preparing the tooling, again, just prepare a package file which lists appropriate type dependencies for them. (And again, optionally a tsconfig listing all the included types to avoid needing to write triple slash lib references)

[mihailik]

I think this perception is due to the use case still not being clear, @weswigham

Let's talk about [2], writing extensions for an existing large app. App's assets are processed and bundled already. Not distributed by a package manager.

Known are precise versions (even forks where applicable) of React, ReactDOM, RX and components. Everything is already bundled, optimised and tree-shaken to the exact shape we wish to expose to the extensions.

For an extension writer it's easiest to have an atomic 1-to-5 files SDK.

As there is no package manager at play, nor module resolution — typescript's rich spectre of module settings, syntaxes and conventions rather gets in the way. The need to spread dozens of DTS files across an elaborate tree of directories is a noticeable pain in the neck.

This use case is quite irrelevant to SharePoint-kind of sites, nor to conventional websites, not too relevant to cloud microservices.

But it is quite relevant to the large apps similar to VSCode or GitHub Desktop. Especially, when those are developed in a corporate environment where you really want to make the toolkit as atomic and blackboxed as reasonable possible.

[mihailik]

Here's an example of a component such a corporate app may bundle and expose as part of the API to the extensions:

The root ag-grid-react/main.d.ts file:

export * from './lib/agGridReact';
export * from './lib/reactCellRendererFactory';
export * from './lib/reactFilterFactory';

and directory with dependent DTS files for that one component:

Index of /lib/
agGridReact.d.ts | text/plain | 820 B | 2017-06-26T14:19:38.000Z
agGridReact.js | application/javascript | 7.2 kB | 2017-06-26T14:19:38.000Z
agReactComponent.d.ts | text/plain | 455 B | 2017-06-26T14:19:38.000Z
agReactComponent.js | application/javascript | 1.55 kB | 2017-06-26T14:19:38.000Z
interfaces.d.ts | text/plain | 886 B | 2017-06-26T14:19:38.000Z
interfaces.js | application/javascript | 102 B | 2017-06-26T14:19:38.000Z
reactCellEditorFactory.d.ts | text/plain | 194 B | 2017-06-26T14:19:38.000Z
reactCellEditorFactory.js | application/javascript | 3.21 kB | 2017-06-26T14:19:38.000Z
reactCellRendererFactory.d.ts | text/plain | 205 B | 2017-06-26T14:19:38.000Z
reactCellRendererFactory.js | application/javascript | 1.5 kB | 2017-06-26T14:19:38.000Z
reactFilterFactory.d.ts | text/plain | 182 B | 2017-06-26T14:19:38.000Z
reactFilterFactory.js | application/javascript | 3.43 kB | 2017-06-26T14:19:38.000Z
reactFrameworkComponentWrapper.d.ts | text/plain | 291 B | 2017-06-26T14:19:38.000Z
reactFrameworkComponentWrapper.js | application/javascript | 2.89 kB | 2017-06-26T14:19:38.000Z
reactFrameworkFactory.d.ts | text/plain | 1.19 kB | 2017-06-26T14:19:38.000Z
reactFrameworkFactory.js | application/javascript | 3.62 kB | 2017-06-26T14:19:38.000Z

[email protected]

[weswigham]

For an extension writer it's easiest to have an atomic 1-to-5 files SDK.

Interesting. Some of the most widely used SDKs are multi-file IDE-included monsters (see the Android and iOS SDKs). Angular even includes a command line tool nowadays. But that's neither here nor there, I suppose.

If you want to freeze the version of the type you ship with your bundle of js and not rely on a package manager, I'd recommend just zipping up the node_modules/@types folder along with everything else you're bundling. (And potentially any included definitions in things which are not @types-y) It's the same bits on disk as it would be inside a supposed single file, except everything already works. (And a folder can get copied just as easily as a file.) Your consumers never need to interact with your files, right? There's no material difference between a file and a folder beyond being able to interact with smaller parts at a time with common utilities (which they're not doing)... What's the actual problem you're trying to solve? What prevents you from shipping a folder of type definition files instead of a single one? Is that problem something we can solve? Are you attempting to discourage them from going in and editing the types by showing it off as a 30000 line monster file when they get curious and open it, rather than lots of smaller, self-contained components? Were you actually looking for someway to specify a library as "blackbox code" which could not have its type information goto'd in an editor?

I'm actively discouraging this because bundling type definition files is, in my option (after seeing how a few people have written their own tools to do it and why they've done it), an antipattern - something people sometimes see as necessary but only because they see it as a way forward to cope with another underlying issue. I don't usually concatenate the entire linux kernel header file set into one file when I want to give it to someone else to have them write a kernel module, to liken the scenario to another environment.

[mihailik]

On practical side, zipping does not work, because extension writers need to use OTHER components too except for those included with the app.

I'll reply on methodology separately.

[weswigham]

On practical side, zipping does not work, because extension writers need to use OTHER components except for those included with the app.

You can list multiple typeRoots in a tsconfig and keep their stuff and your stuff separate.

[mihailik]

@weswigham your suggestion is to deploy DTS typings as a tree structure of directories/files

The human maintenance cost and build drag of that solution is far from negligible.

ZIP suggestion you've mentioned is unfeasible in any sane way except for trivial demos. As soon as you just consider version upgrades the problem domain explodes (timestamp precision, partial copies, incompatible unzip syntax running with CI...)

NPM/YARN has versions, flavours and bugs. They also require webpack/rollup to peruse -- those come with their own high complexity and steep learning curves. All that incurs a permanent maintenance tax roughly O(n) per DTS file.

Build time drag in both of these approaches is O(n). Querying files and directories is a major kernel call, which for small files bluntly outweighs the cost of content reading.

Please see an example of unbundled DTS output in a popular component used in many corporate environments above: #17196 (comment) -- a sundry of tiny files with no clue which file contributes to the external API and which is internal stuff dumped by TSC because it can.

Bundling of DTS files collapses O(n) to O(1).

[mihailik]

@weswigham to highlight: this is not my opinion. It's one of the practices EXISTING out there.

Microsoft Visual Studio Code bundles huge 5K lines DTS in one file monaco.d.ts.

Note how it's compiled out of huge list of other files, look at ICommonCodeEditor in monaco.d.ts and the same ICommonCodeEditor originally living in editorCommon.ts (which is a module, not a script).

@jrieken can you give a brief overview, what do you do to you generate monaco.d.ts?

[jrieken]

@jrieken can you give a brief overview, what do you do to you generate monaco.d.ts?

That is a very special process to which @alexandrudima can answer questions. See these two files to find you ways into it: https://github.com/Microsoft/vscode/blob/master/build/monaco/api.ts#L364, https://github.com/Microsoft/vscode/blob/master/build/monaco/monaco.d.ts.recipe#L1

[mhegazy]

So the request here is to enable export as namespace foo to work inside an ambient module declaration? i.e.:

declare module "foo" { 
    export = {};
    export as namespace Foo;
}

[mihailik]

@mhegazy yes, great syntax!

And the second part, declare import would be desirable as a way to produce such code as output via --declare.

[mhegazy]

Automatically closing this issue for housekeeping purposes. The issue labels indicate that it is unactionable at the moment or has already been addressed.

[mihailik]

Thanks for the heads up @mhegazy . Can you please clarify 'unactionable'?

Your syntax suggestion doesn't seems to have any structural issues. Is it closed due to some undisclosed internal discussion?

[mhegazy]

Thanks for the heads up @mhegazy . Can you please clarify 'unactionable'?

if my script sees Needs More Info it closes it after 2 weeks.