TypeScript integration

[shirakaba]

I don't see any TypeScript definition files, nor an @types/boardgame.io module on npm.

1. Would addition of TypeScript definition files be welcomed (this would be totally non-intrusive, and can be done incrementally, but would have to be maintained every time any JS API updates)?
2. Would actually migrating all the JS source to TypeScript be welcomed (the type definitions would be future-proofed, and we could use any level of ES features throughout the code, transpiling down to whatever target necessary; but it may alienate contributors only knowledgeable of JS)?

[nicolodavis]

I'm open to adding typescript definition files if you want to send me a PR. I would prefer that the codebase stays in JS for now, though.

[chrisheninger]

@shirakaba if you'd like a starting point– @felizardo has a file in his turnato repo:

https://github.com/Felizardo/turnato/blob/master/%40types/boardgame.io/index.d.ts

There was also a little discussion around TS when discussing the Angular client: #131

[shirakaba]

@chrisheninger That's invaluable, thank you!

Would a single-file declaration describing the whole codebase via many declare module "x" {} statements be preferable to declaration files alongside each JS file, then?

[shirakaba]

For now, I've had to do declaration files alongside each JS file because I'm unsure how else to make test files for the typings. The test files are similarly littering the UI folder. I've also been incrementally adding the types into one global declaration file so that we have both options.

You can see my progress on my fork, on the shirakaba/typescript branch.

I've so far typed all the src in the UI folder, but still have a few issues to resolve...

The foremost issue is that I don't know how to correctly type Logo and Card (which both use stateless component functions – what's more, they're assigned to const variables, and Logo adds in a default export to increase the headache further). Some help there would be appreciated.

• Generally, I'm surprised that the UI components have so many optional features, and that many of them expect just a single child rather than multiple children.

• In the ui/token.js, defaultProps.template references the Square class but doesn't construct an instance of it; is this a bug?

• What templates may Token accept? Just Squares?

There's plenty more work to be done, still. Would welcome any extra pairs of hands.

[shirakaba]

The foremost issue is that I don't know how to correctly type Logo and Card (which both use stateless component functions

I figured it out with a little reference to https://github.com/piotrwitek/react-redux-typescript-guide#component-typing-patterns and some guesswork.

I now have Logo and Card typings that permit all the patterns used in logo.test.js and card.test.js, but I've stumbled across a question for more experienced React developers:

Should each of these UI elements accept all props specified for their element type, e.g.: HTMLAttributes<HTMLDivElement>? Or should consumers of the library be restricted to inputting only the props specified in propTypes? I guess they should accept all relevant HTML attributes, to be in line with is permitted through use of the (untyped) JSX..?

[shirakaba]

Typings for server folder provided now. Some left as any as I need to type the core before I can get more specific with some values.

[scally]

@shirakaba Is the intent that these typings ship with boardgame.io instead of in DefinitelyTyped? I see that they're in the repo now in your fork, but wasn't sure if that was a temporary thing as you worked on the typings.

[shirakaba]

@scally I'd ideally prefer to ship them with boardgame.io directly, so that maintainers can keep the JS typings definitely in step for each given version of the repo. Otherwise, it becomes unclear which version of the typings one would need to install to match, say, v1.2.3 of the repository (a common headache for TS devs).

More than anything though, I'm not sure how to concatenate individual typings into a single file; there are a few things like default exports and modules that I don't know how to handle in the form of a single-file declaration – which would also impede me from launching it as a DefinitelyTyped project.

I'm not sure what the linting settings for DefinitelyTyped are, either – likely I'd have to rewrite a bunch of stuff.

Still open for discussion about how best to organise the files (ideally, they should only end up being seen in dist, but yes, I'm just writing them in the src folder for now before thinking about the final repo structure).

[shirakaba]

I've just finished making typings for the core module (with a small number of TODO notes here and there for review) – whew!

[shirakaba]

I've now written typings for all the JS in src (having just finished typing the client module). 🎉

On reflection, I've just noticed that I could write typings for the four CSS files, too 🤔 could try, I guess.

Blockers:

• I'll need to do another pass to add the (fairly insubstantial) typings tests for core and client, but the hard part is done now, at least.

• I should try applying the typings to some/all of the example projects to see if anything comes up.

• We need to discuss the structure for where to put all these files. I'm happiest just leaving the .d.ts files directly alongside their .js counterparts, because it makes the development experience much better (you don't need to wade through thousands of lines to find the typings you need), and keeps the modularity intact. The other option is to concatenate the typings into one mammoth file, but I'm not actually sure how to do that.

• I'll need to sign the Google contributors agreement before making any pull request; will need to speak with my company about this first.

• The pull request will need review particularly by contributors who really understand how React and Redux should be typed (I'm sure the Redux typings won't be perfect).

[nicolodavis]

Another option that would eliminate most of the blockers would be to keep it in a separate repo for now until we reach v1.0 and have a stable API (and merge it in then). I worry that the lack of TypeScript expertise and the ever changing API during the alpha stage would make development inconvenient for developers in this repo.

[shirakaba]

@nicolodavis Sorry for the late reply.

keep it in a separate repo for now until we reach v1.0
Is there a timescale in mind for this? Naturally I worry about my types drifting out of sync and become obsolete if there's too much change to catch up with, but I'll do my best if TypeScript integration is a real goal.

For now, I'm happy to fill up a separate repo with TypeScript examples and refining the types.

It's looking like the path of least resistance may end up being creating a single-file global declaration after all (as the JS is packaged rather than left as individual source files), so we may even have the option of going through DefinitelyTyped in early stages before merging in post-alpha, although that's rather unattractive for me (the typing and linting strictness is of insanely high standard for DefinitelyTyped, and I have dozens of files to get in line).

I worry that the lack of TypeScript expertise and the ever changing API during the alpha stage would make development inconvenient for developers in this repo.

I'll admit I can see refactoring the JS becoming harder if types need to be kept in step each time.

[nicolodavis]

Thanks @shirakaba!

Yeah, I do think this will be the easiest way to make progress going forward until we hit v1.0 (I'll probably upload a roadmap for this so people have a general sense of the timeline). In the meantime, once you're satisfied with your typings, let me know so I can link to your repo from the documentation here.

[shirakaba]

I've now combined all the individual-file declarations into one whole-repo index.d.ts modules declaration file (same structure as your original):

https://github.com/shirakaba/boardgame.io/tree/shirakaba/typescript-examples

yarn add https://github.com/shirakaba/boardgame.io.git#shirakaba/typescript-examples

However, I'm getting the classic packaging cache problem: facebook/react-native#4968 and so can't check whether the typings actually work when distributed as one single file.

@felizardo Did you experience any similar problems when setting up the TicTacToe example? Does your example still compile if you swap your index.d.ts file for mine? Instead of the declared paths being boardgame.io/core, should they include the dist folder, i.e.: boardgame.io/dist/core?

Edit: I've tried making a whole new project, separate from the repo, but that's facing the same error, too :( I'm blocked now.

[scally]

@shirakaba Just in case the DefinitelyTyped option could save you some hassle, I researched some of the pain points you mentioned.

Otherwise, it becomes unclear which version of the typings one would need to install to match, say, v1.2.3 of the repository (a common headache for TS devs).

Some type definitions include a version at the top indicating which release it matches, like this:

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/react/index.d.ts#L1

I'm not sure what the linting settings for DefinitelyTyped are, either – likely I'd have to rewrite a bunch of stuff.

It looks like it might be defined per-package like this:

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/react/tslint.json

Also, tslint (at least its extension in VSCode) can autofix a lot of problems without complaint on save.

[scally]

Just out of curiousity @nicolodavis is there a particular thing keeping you from wanting to use Typescript for this project? It seems like it might be the path of least resistance to integrating types here.

[shirakaba]

One thing to add/clarify: I can't get a single-file index.d.ts to work with the bundled js (so DefinitelyTyped is out of the question unless someone can figure it out to unblock me), but the typings would work just fine if the project were to distribute its individual .js files and distribute the respective .d.ts files alongside those.

[shirakaba]

Thus, DefinitelyTyped could be considered if the js source were to be distributed as individual files rather than bundles.

[nicolodavis]

For me, it's just a lack of knowledge of Typescript that wants me to keep it separate for now. That and the fact that I think the somewhat fluid API at the moment will be harder to modify with more things to keep in sync. I'm totally open to getting it in after we reach v1.0.

Which is also a reminder for me to get started on documenting the roadmap to v1.0.

EDIT: Roadmap link

[shirakaba]

@nicolodavis Thanks for that roadmap just now.

[vdfdev]

Oh, I am also maintaining my own .d.ts for my project, lets see if we can collaborate for a single .d.ts :)

[shirakaba]

@flamecoals My declarations were written a year ago, and I didn't maintain them further, so I'd recommend you just continue with your own. There were some issues with my typings in any case:

1. While the typings were correct for each individual module, I couldn't get them to correspond to the Rollup build (see my rant about TS libraries below).
2. The schema for the state store was pretty unclear from the source at the time, and Redux typings are pretty much the hardest part of TS to begin with, so I don't guarantee 100% accuracy there.
3. The code-base isn't stable yet, so writing the typings is a constant game of catch-up with the JS developers. I also found occasional errors in the JS and had to make executive decisions on the typings in some cases.

I personally think that the way forward is to migrate the codebase to TS (this would only take a weekend) and distribute it as a library of modules, leaving the bundling to the consumer (there is no way to auto-generate declarations for a bundle of TS modules). However, Nicolo is only open to a migration upon reaching a stable version. And I will admit that TS is a double-edged sword when it comes to Redux-based projects; the Intellisense is great, but the typings can be hard to set up.

I see that there's a Gitter. I'll hang out there and get re-acquainted with this project :)

[vdfdev]

Thanks. I will do my best to make this reusable by others, even if we need to change over time. I will see if there is any way to add automated tests to see if the types broke.
I agree with you that ideally we would use TS everywhere, and I think that having clear interfaces would make my life easier whenever I wanted to contribute to bg.io. I struggle a lot to understand the interfaces right now.
But again, Nicolo did most of the work in this framework so I respect his decisions.

[nicolodavis]

I'm more open to TypeScript these days having gained a bit more familiarity with it.

I'm not opposed to converting some of the code in src/core to TypeScript and adopting a gradual typing approach to converting the project over. My main concern is that it might reduce the pool of potential contributors.

If either of you has some time, I'm happy to look at a PR that adds some types to src/core/flow.js and src/core/reducer.js. These are the two places that will benefit the most from having types.

[shirakaba]

I'm more open to TypeScript these days having gained a bit more familiarity with it.

Oh, cool :)

My main concern is that it might reduce the pool of potential contributors.

@nicolodavis This is a valid concern, although the other side of the coin is that it may attract TypeScript devs (and there are quite a lot of us in the React Native community). I've always held back from contributing because I feel a bit powerless to help if typings aren't in place to make clear all the contracts. I'd much rather the IDE enforce my typings than have to hope that the documentation is up-to-date and sufficient.

adopting a gradual typing approach to converting the project over

Incremental adoption is certainly possible, but presents some problems:

• Having certain modules written in TS would help repo contributors, but wouldn't directly help users of the library (because it's shipped as a bundle, which would remain untyped). Hand-writing and hand-maintaining a declarations file for the bundle is undesirable, so we'd need to consider shipping the individual sources, alongside their declarations files, for consumption as a TS library (where the consumer would be in charge of the bundling).
• We'd need to insert a "transpile TS sources to JS" step before the Rollup build (which is fine).

Incremental adoption does make sense in terms of reviewing PRs and reducing merge conflicts from active PRs.

I'd love to try, if we can clarify the issues of bundling and shipping.

[nicolodavis]

Right now Rollup generates a few different files for various subpackages (one for each file in the package directory).

Regardless of whether we convert some (or all) the code to TS, we'll still need to generate these JS files in the final NPM as well as a minified bundle that users can insert via a <script> tag.

I'm thinking that we can just convert all the code that corresponds to one subpackage to TS as a pilot and see how it goes. core.js is a good candidate for initial conversion. This will just benefit repo contributors initially (i.e. no need to maintain a declarations file for now) and the idea is that we'll eventually convert everything over so that it can be consumed as a TS library.

1. Do the TS sources go into a separate NPM package? We'd still need to keep the current NPM with the JS code as-is so that JS users can consume them.

2. Are you saying that we don't need a declarations file if everything is in TS? I'm assuming that we'd still need to export the same types that correspond to the external interfaces of the library.

[shirakaba]

Regardless of whether we convert some (or all) the code to TS, we'll still need to generate these JS files in the final NPM as well as a minified bundle that users can insert via a <script> tag.

I see, so that's the use case of the bundle. Okay, we'll need to research solutions for offering typings post-bundling.

I'm thinking that we can just convert all the code that corresponds to one subpackage to TS as a pilot and see how it goes.

Can do.

Example TS library (disregarding bundling for now)

Do the TS sources go into a separate NPM package?

To answer your two questions, here's how I prepare my own TS libraries (node_modules folder omitted for brevity):

.
.
├── LICENCE.txt
├── README.md
├── dist
│   ├── DirectoryLister.d.ts
│   ├── DirectoryLister.js
│   ├── DirectoryLister.js.map
│   ├── Rsyncer.js
├── package-lock.json
├── package.json
├── src
│   ├── DirectoryLister.ts
│   └── Rsyncer.js
├── test
│   ├── index.spec.ts
│   ├── mocha.env.js
│   ├── mocha.opts
│   └── setup.ts
├── tsconfigs
│   ├── base.json
│   └── src.json
└── ambientTypings
    └── child-process-wrappers
        └── index.d.ts

• Both JS and TS sources live in src.
• ambientTypings is only necessary to declare any typings that would be set on global scope. If the whole codebase is modular, then we don't need this.
• The following folders would need to be distributed on npm: dist and ambientTypings.
• All sources compile to dist. Any source that derives from a TS file will get a .d.ts declarations file; sources that derive from JS won't, and a TS developer trying to import it would have to import it as any type.
• In my libraries, there is no bundle step; that is left to the consumer. We can look into this.

Benefits of writing source in TS

Are you saying that we don't need a declarations file if everything is in TS?

I'm saying that the .d.ts files for each module would be automatically produced and would thus always stay up-to-date (and more importantly, be correct!). We'd still need to somehow produce declarations for the bundle, however (which I comment on below).

Do the TS sources go into a separate NPM package?

If we write everything in TS, we'd have two options:

1. distribute the .ts sources and let the developer import those. As nice as this sounds (i.e. devs can inspect the TS implementation), I've always found the TS compiler complaining about importing .ts files from node_modules (it's normally ignored in tsconfig.json), so I'd recommend against this approach.
2. transpile each .ts file to its corresponding .d.ts, .js, and (optionally) .js.map files. In this approach, I would typically expect the developer to be importing file-by-file rather than importing a bundle, but we can explore ways to provide declarations for a bundle.

In practice, TS libraries always go with option 2 (as I've illustrated in my tree structure).

Distributing declarations for a bundle

The moment we produce a bundle from all our transpiled JS sources, our typings no longer match the way that the developer will be importing the modules. So either we need to write such typings or find a plugin for the bundler to do it for us automatically. This may actually be easier nowadays than I've historically thought. Here's some quick research:

Providing definitions for the bundle

Manual

Source: https://stackoverflow.com/a/41016522/5951226

boardgame.io/index.d.ts

Provide a single file with an import and declare statement for each package from packages entering the bundle, thanks to the typeof operator.

import * as _core from './core';
import * as _client from './client';
// etc.

declare var core: typeof _core;
declare var client: typeof _client;
// etc.

... Which is perfectly low-maintenance. I wasn't aware of this method as typeof is a relatively new operator that I haven't got used to using. If this is all it would take, I'd be happy with this. We'd just need a script to copy this hand-written .d.ts into the right place – our dist folder, if that's where the corresponding index.js would be – as-is before publishing to npm.

I guess I'd favour this method if it works? Mainly because it's so unmagical, not complicating the build process.

If boardgame.io requires ambient declarations, we could potentially shove them in here rather than distributing ambientDeclarations, too. May be necessary if we're bundling the code, anyway.

Automatic

Source: https://stackoverflow.com/questions/41931610/generate-bundle-typescript-definition-file-with-webpack

It looks like there are several (see link) Webpack plugins for automatically producing .d.ts files for the bundle; there may be equivalents for Rollup.

const NpmDtsPlugin = require('npm-dts-webpack-plugin')

module.exports = {
  // ......
  plugins: [
    new NpmDtsPlugin()
  ],
  // ......
}

Rollup is totally unfamiliar to me, however.

[nicolodavis]

If we write everything in TS, we'd have two options:

1. distribute the .ts sources and let the developer import those. As nice as this sounds (i.e. devs can inspect the TS implementation), I've always found the TS compiler complaining about importing .ts files from node_modules (it's normally ignored in tsconfig.json), so I'd recommend against this approach.
2. transpile each .ts file to its corresponding .d.ts, .js, and (optionally) .js.map files. In this approach, I would typically expect the developer to be importing file-by-file rather than importing a bundle, but we can explore ways to provide declarations for a bundle.

Option 2 sounds good. That would keep our existing NPM package relatively unchanged, except that each JS file now has a .d.ts as well.

About the bundle, I'm OK if we don't ship any typings for it. We'd then force TypeScript devs to import our modules (rather than the bundle, which is only used by JS devs that want a quick prototype without Webpack). This is a reasonable constraint to place on TS devs, who probably have more sophisticated build environments to start out with anyway.

[nicolodavis]

So, just to make sure we're on the same page, our next step is to convert the sources that feed into core.js to TS? This will result in no change to the NPM except for the addition of core.d.ts, which will allow TS devs to have typing when they import this module.

They'll still be able to import the other modules as type any, correct? We'll eventually convert them over to TS as well if the first conversion works well, so this would be a temporary situation.

[jasonharrison]

#431

[Oliboy50]

thanks @jasonharrison

it seems that the PR you're linking only allowed the boardgame.io to host TS files

in term of typings, it won't help much with our TS projects yet

hopefully types are going to pop faster now that you've started the TS integration, good job 👍

[jasonharrison]

@Oliboy50 Yup, my PR was so BGIO can begin to use TypeScript :)

I actually am not sure how to tell NPM about our declaration files (blah.d.ts)...
I am getting confused about how to get our declaration files recognized by NPM because we use Rollup and aliases and such.

If you're familiar and you have the time, maybe you could help me with this?

[nicolodavis]

This is more or less done now. The only thing that remains is to export (tsc generated) type definitions in the NPM and then users will have types for the external interfaces of boardgame.io.

[sparr]

@nicolodavis looking forward to it!

[delucis]

This is available now — if not documented yet.

General types can be imported directly from boardgame.io, for example:

import type { Game } from 'boardgame.io';

interface G {
  score?: number;
  deck: string[];
}

const game: Game<G> = {
  name: 'my-game',
  // setup’s return will be checked against your G type
  setup: () => ({
    deck: [],
  }),
}

[crhistianramirez]

How exactly are types being added?

This is what GameConfig.setup is typed as here

setup?: (ctx: Ctx, setupData?: any) => any;

but in dist/types/src/types.d.ts it shows up as simply

setup?: Function;

Which is not nearly as helpful

[delucis]

@crhistianramirez Which version are you using? This was fixed in 866e897 and released in v0.39.4.

[crhistianramirez]

@delucis that is the version I have downloaded.

I downloaded zip of the release and inspected the contents. Not seeing it there yet. Thinking it might not have been built before it was published

[delucis]

Ah, that ZIP is just the source code, not the built release, but taking a look, the release does seem to be missing this update, so maybe your hunch is correct:

https://unpkg.com/browse/[email protected]/dist/types/src/types.d.ts

@nicolodavis Is it possible Typescript did an incremental build when you released and didn’t include these updates or that the TS command isn’t auto-run on release?

[nicolodavis]

The TS command is run by Rollup, so I'm not sure what's broken here. Let me push another release and see if that resolves it.

[nicolodavis]

@crhistianramirez Can you try 0.39.5?

[delucis]

Looks good, Nicolo!

https://unpkg.com/browse/[email protected]/dist/types/src/types.d.ts

[nicolodavis]

Also, do you guys think that Game would be a more intuitive name than GameConfig?

[delucis]

Perhaps? It’s actually the same interface as that returned by the Game function right?

[crhistianramirez]

Looks good!!

Yes, I agree Game is more intuitive.

[will-hart]

The new typescript types in 0.39.6 seem to work really well for me... 💯!

I am having some issues with Server types. Various combinations of importing the server object failed to get typings, and only require as per the docs seemed to work (although the import was typed as any). I see these were discussed a bit in #576, so maybe these haven't been implemented yet?

[delucis]

@will-hart Yes, the server module is bundled slightly different from the others as you saw discussed in #576, so its types aren’t exposed correctly when doing import { Server } from 'boardgame.io/server', and that hasn’t been fixed yet. The types are included in the build, (e.g. at boardgame.io/dist/types/src/server/index.d.ts), so I’ll see if I can figure out a quick fix.

[delucis]

@will-hart This should be fixed once #622 is merged and released.

[coyotte508]

I can't import boardgame.io/client:

ERROR in <xxx>/node_modules/boardgame.io/dist/types/packages/client.d.ts(1,24):
1:24 Cannot find module '../src/client/client'.
  > 1 | import { Client } from '../src/client/client';
      |                        ^
    2 | export { Client };
    3 | 

It seems dist/types/src/client is missing:

[delucis]

@coyotte508 I think this is because the client source code hasn’t been converted to Typescript yet. I’m working on some changes to those files at the moment, so I’ll try to make that conversion while I do.

[coyotte508]

Ok :)

In the meantime import boardgame.io/dist/cjs/client seems to work

[nicolodavis]

#622 is released in 0.39.8.

[brebory]

I was running into some issues using the PluginPlayer plugin in 0.43.3 while using TypeScript. Specifically, when trying to create my strictly-typed game definition, I was unable to publicly import the derived PlayerPlugin type to extend Ctx.

Specifically, I tried to define my game like this:

export const GameDefinition: Game<GameState, Ctx & PlayerPlugin<PlayerState>, CustomSetupData> = {
  // game definition elided
}

This definition causes my CtxWithPlugins type to properly include the player property with the PlayerAPI<PlayerState> type, which in turn makes my custom PlayerState properties pass type checking when using ctx.player.get and ctx.player.set.

However, the PlayerPlugin type is not a public export of boardgame.io/plugins. In order to gain access to PlayerPlugin I had to use the following import:

import { PlayerPlugin } from 'boardgame.io/dist/types/src/plugins/plugin-player';

Have I made an error, or is this something that can be fixed?

IMO it would make more sense to include PlayerPlugin and other such interfaces in the boardgame.io/plugins package. If this should be fixed, I'd definitely be open to creating a PR if that would be helpful.

[delucis]

While we still need better documentation of usage with TypeScript, current releases of boardgame.io are bundled with fairly complete typings, so I believe this can be closed.