💌📯 Updates #130
Comments
November Catch-up
I have two not yet published blogs posts on the TypeScript product blog, one on our usage of GitHub Actions in the Playground in the repo
The playground self-documents a lot of these in a set of examples Long termOn the website v2 side, I've created larger roadmaps which talk to some of the long-term vision on different parts:
Work in Progress:
Shipped Infra I'm happy with:The new website has a solid amount of automation:
Thanks
|
Website NavigationI've focused my work with three main goals since the last update:
I'm going to focus this update on the navigation aspects of the site. Let's look at a few different places we have navigation and talk about why it was designed this way Site NavigationI tried to take a mix of modern native mobile app design, and website best practices to focus on three main audiences:
The first two tend to have similar needs in terms of what they want for site navigation, but mobile devices themselves should be treated quite differently both from a user's intent and how we should present navigation. How does it look on desktop?
How does it look on mobile?
Why the drastic change?Navigation on desktop tends to take a lot of design cues from headers in print design which aims to follow the path of reading:
But mobile really disrupted that from two factors:
This moved the design frames of references from one loosely based on fitt's law to In designing for mobile, I had to really narrow the priorities for people to two things. I wanted people to easily get to search, and to go to the root of the documentation. In my experience building websites for developer documentation, most people are using mobile as a reference or to quickly show something to someone. You can see here that we barely get into the natural section in search
Then just get comfy when the navigation bars appears
( thanks Thuy Gia Nguyen for the heatmap) With the navigation designed, I applied the same forms of navigation hiding which mobile browsers do. The navigation will only show when you start to move up. It also keeps track of display curves for phones like the iPhone X and my Samsung Note, you can see in this GIF when the browser includes a flat edge at the bottom then the navigation buttons lay flat.
I still have a few questions to answer though:
I expect to take some cues from docs.microsoft.com. FootersI believe in mega-footers. A footer is the end of the information you were actually looking for and generally represents three things:
With the TypeScript footer, to handle the I took a look at the most popular documentation pages on the website and included them to make sure that people would have easy access to them. Their order is in popularity. To hit the second, I wanted to raise the visibility of the playground code samples. These provide a very focused explanation of many parts of how TypeScript works and aren't easy to find as a sub-navigation element inside the playground. When viewport to the site is large enough, and JavaScript is enabled then the Code Samples link switches to a popover which echos the design in the playground.
For mobile, I didn't need to do anything content-wise. I don't think there's any value in making it more focused. Design wise I only made a few tweaks but it felt fine in both. Internal NavigationThe other place where we have navigation is between pages. When you link into a handbook, we want to show a navigation between related documents.
This is a pretty solid design pattern to work within, and one that's common on documentation sites. I took care to make it obvious which navigation section you're in and Gatsby made it so fast that it doesn't feel like you're navigating between web pages. The tricky thing was making it feel right on keyboard, which I'm really happy with. No runtime dependenciesA lot of what you're seeing in these screenshots looks like a lot of complicated JavaScript code, but it's not. Apart from search which comes from Algolia, the v2 site has zero JavaScript dependencies at runtime, and is a static HTML site to an azure blob storage account. The interaction patterns are built from first building the site with JavaScript disabled, and then JavaScript to handle some of the extra is added in at runtime. I use the This pattern works really well for a medium sized site with only 1-2 core contributors. |
Interactive Codeblocks
It's been a slow week on the web infra and docs world. The tsconfig docs are pretty close to being wrapped up now. However, the main reason for the slow week has been that I was on my first Definitely Typed rotation last week, and I'm covering mornings this week. In-between DT work, I've been wrapping my head around twoslash improvements. Twoslash is a mini-typescript sandbox environment where you can control the compiler and the output via comments. It's part of a larger roadmap #120 but it's basically the tool doing the heavy lift. For example: // @declaration: true
// @showEmit
// @showEmittedFile: index.d.ts
/**
* Gets the length of a string
* @param value a string
*/
export function getStringLength(value: string) {
return value.length
}Tells the compiler with the first three lines:
So the user-facing code sample would be: /**
* Gets the length of a string
* @param value a string
*/
export declare function getStringLength(value: string): number;This means the code sample a user see is a Twoslash Features
For us the critical stuff is showing errors, highlighting parts of source code and showing the results of transpilations. The nice to haves are things like handling imports in a single file, and playgrounds. The advantages for the site are:
This last point is particularly useful because of how TypeScript type inference works with narrowing or widening - the same variable identifier could be very different objects in different parts of the same code sample. Highlights for you, and you, and youWhile it's great for us to highlight the bits we think are important in a code sample, the ideal state is that our code samples provide a way to highlight the code at all the identifiers in that sample. This is one of the features I've been working on this week:
While not 100%, you can feel all of the pieces coming together:
Ecosystem AdoptionNearly all of these twoslash features aren't problems unique to TypeScript, twoslash uses the language server protocol to extra all of these annotations - meaning essentially any language can re-create this infrastructure in their docs (and hopefully after seeing it on the TypeScript v2 site, they will!) I've also been building twoslash support out as a series of modules to work with Gatsby, so I would hope this makes it easy for folks to use adopt twoslash for their sites when they are writing about TypeScript. |
|
TSConfig ReferenceOne of the completely new features in the new website is the TSConfig Reference. Since the last update, this page is now at a "good enough" stage that it's pretty publicly available. The design is a WIP, but the copy is 👍. https://www.typescriptlang.org/v2/en/tsconfig The TSConfig Reference has a few goals:
De-mystifying Compiler FlagsPrior to the TSConfig reference, the canonical source of information on the available flags and how they work is the overview of the command line flags for TypeScript. This is/was a table of tweet-sized descriptions of most CLI flags, of which the majority are also tsconfig options. You would then likely need to either get information from the release notes which introduced that flag, or to search the internet for that exact flag name to get more info. This documentation structure is pretty logical because the TypeScript codebase keeps both tsconfig and CLI flags in the same data-structure, which has made it hard to extract either for documentation purposes. My first port of call at starting this page was to use the typescript API to extract all of the compiler flags and to generate a JSON dump of all the compile flags. This comes to about 100+ flags today. export type CompilerOptionName = "help" | "watch" | "preserveWatchOutput" | "listFiles" | "listEmittedFiles" | "pretty" | "traceResolution" | "diagnostics" | "extendedDiagnostics" | "generateCpuProfile" | "incremental" | "locale" | "all" | "version" | "init" | "project" | "build" | "showConfig" | "listFilesOnly" | "target" | "module" | "lib" | "allowJs" | "checkJs" | "jsx" | "declaration" | "declarationMap" | "emitDeclarationOnly" | "sourceMap" | "outFile" | "outDir" | "rootDir" | "composite" | "tsBuildInfoFile" | "removeComments" | "noEmit" | "importHelpers" | "downlevelIteration" | "isolatedModules" | "strict" | "noImplicitAny" | "strictNullChecks" | "strictFunctionTypes" | "strictBindCallApply" | "strictPropertyInitialization" | "noImplicitThis" | "alwaysStrict" | "noUnusedLocals" | "noUnusedParameters" | "noImplicitReturns" | "noFallthroughCasesInSwitch" | "moduleResolution" | "baseUrl" | "paths" | "rootDirs" | "typeRoots" | "types" | "allowSyntheticDefaultImports" | "esModuleInterop" | "preserveSymlinks" | "allowUmdGlobalAccess" | "sourceRoot" | "mapRoot" | "inlineSourceMap" | "inlineSources" | "experimentalDecorators" | "emitDecoratorMetadata" | "jsxFactory" | "resolveJsonModule" | "out" | "reactNamespace" | "skipDefaultLibCheck" | "charset" | "emitBOM" | "newLine" | "noErrorTruncation" | "noLib" | "noResolve" | "stripInternal" | "disableSizeLimit" | "disableSourceOfProjectReferenceRedirect" | "noImplicitUseStrict" | "noEmitHelpers" | "noEmitOnError" | "preserveConstEnums" | "declarationDir" | "skipLibCheck" | "allowUnusedLabels" | "allowUnreachableCode" | "suppressExcessPropertyErrors" | "suppressImplicitAnyIndexErrors" | "forceConsistentCasingInFileNames" | "maxNodeModuleJsDepth" | "noStrictGenericChecks" | "useDefineForClassFields" | "keyofStringsOnly" | "plugins";Having a data-dump is a good first start, but some of these flags are obviously CLI only, for example: To map these domains, there is a file called This file tracks:
From there the next step is to generate a markdown file for the site to work with. This is done in Real use-casesThe TSConfig reference is a perfect place for building out the features of the twoslash because it requires making code samples which have all sorts of emits, compiler failures, and configuration options. A great example of this is the
Low Barrier to ContributeEach compiler option is a unique markdown file which will eventually be concatinated into a single file. This means a contributor can easily make isolated changes to just the file they're interested in. Because it's plain old markup and twoslash is reasonably simple then we can get a lot of bang for a small amount of code. Being a series of markdown files means that tooling like Danger can run spellcheckers, and linters against each file individually to ensure we're shipping great docs. Ready for TranslationsThis project was built with internationalization from day one. I've been making a really bad language attempt at supporting a l33t speak dialect under "vo" to prove the system: https://www.typescriptlang.org/v2/vo/tsconfig#module The idea is that the community can add translations option by option, and if they haven't included one yet then the english version is the fallback. So while it might be simple - it works well and it's the first part of the v2 website to hit all three of the targets for the internationalization roadmap. Meaning it's ready to go it you want to start translation to a language you know. Docs are in the README |
State of the v2nion
type State = "playground" | "internationalization" | "dev-pages"I've been plugging away occasionally on the last two weeks on the new Playground, and some developer pages for the node modules which I needed for the website. I'll cover those later, but now that we've hit the 3.8 beta, it's time for me to slow down on the web infra to work on some compiler bugs. So, unless I feel like deep-diving on a particular topic there may not be an update in 2 weeks time. LocalizationI originally started working on supporting multi-lingual content for the site about 5 months ago, when I was doing my initial work on forking TypeScript-play to be the new TypeScript Playground. The site was still in design phase, but it felt like an achievable goal, if done in small steps regularly while any type of content was being made. That said, localization for v2 actually came with three interesting problems:
PlaygroundI gave a TL:DR on the main changes to the TypeScript Playground v2 back in Novemeber, but now we have Playground v3. This has about 90% of the features of the v2 playground,
The playground was the first thing I designed for the TypeScript v2 website, and so it's nice to finally see and use it. I reflected earlier on twitter about how design work can feel like a rough plan, because the moment you start iterating - you have a much better set of assumptions about how something works. Dependencies For OthersI mentioned the Sandbox above, I've taken some of the larger dependencies and given them their own section of the website. Both of these are really useful tools if you're working with TypeScript on the web, so I wanted to highlight tooling we've made as quick overviews, I'm sure there are a lot of other cool modules the TypeScript team have shipped, so I want to try highlight how to work with the TypeScript compiler API and some other useful tools. Current Sitemap
My aim once I get back to the site:
|
Why Gatsby?The TypeScript v1 site is a jekyll website, and Jekyll packs a lot of power into a small tool. Jekyll is really great way to build static websites, but it's built to work for small websites of around 1-20 pages. You can feel this in how they treat templating (liquid, which is a logic-less templating engine), how they treat the data modelling internally (there are only really 'Posts' & 'Pages') and how the tool is set up to work with a specific folder structure. At Artsy, where I worked at before TypeScript, we had started to hit the limits of working within Jekyll at around 200 blog posts and a lot of custom pages, and we were exploring different tools to use as a writing environment. In the process I looked deeply into Gatsby, and concluded that it was the right abstraction for building static sites. What Makes Gatsby UniqueWhat makes Gatsby unique among static site generators is this idea that it adds an extra step to the process. In a normal static site generator, you would more or less directly map files to their output: const files = fs.getDirSync()
const htmlFiles = files.map(makePage)
htmlFiles.forEach(html => {
fs.writeFileSync(filename, html)
})Gatsby on the other hand does something a bit more like this: const setupSite = () => {
const files = fs.getDirSync()
const data = files.map(makePage)
graphQLServer.add(data)
}
const createBlog = () => {
const pages = graphQLServer.query("{ pages { title, text } }")
const htmls = pages.map(renderComponent)
htmls.forEach(html => {
fs.writeFileSync(filename, html)
})
}
setupSite()
createBlog()Gatsby adds a GraphQL API which sits in-between the setup of the data and the generation of files in your static site. This abstraction providing a very strong separation of "setting up the site" vs "representation on the file system" which I've found makes it easier to reason about what's going on internally. What does this look like in practice? It starts at
It's a few more steps then Types For ToolsThe TypeScript support in Gatsby is good, and improving as they start to port their own codebase to TypeScript. When I first started, I shipped a few If you're familiar with React, and clicked through into the TSConfig Template - you might have been a bit surprised by the somewhat unorthodox usage of React. I'm using React as a templating language, and not as a reactive UI framework, the site never use a My goal is that the TypeScript v2 website, with all its complexity, needs to be understood with the least amount of abstractions possible. It should not be too surprising, but vast majority of the TypeScript compiler team have a compiler background, and don't really do web development. To ensure that they can contribute, and understand the codebase I'm aiming to use Gatsby and React to get as close to HTML + CSS as possible. One way to do that is to separate the generation of HTML + CSS, from any extra JS which happens at runtime. This means almost every component in the site conforms to this pattern: // JS imports
import React, { useEffect } from "react"
import { Layout } from "../components/layout"
// Style
import "./tsconfig.scss"
// The main React component
const TSConfigReferenceTemplateComponent = (props: PropTypes) => {
useEffect(() => {
// code which happens when the page has finished loading
})
// creation of static HTML via React + JSX
const post = props.data.markdownRemark
return (
<Layout locale={props.pageContext.locale}>
<div className="tsconfig">
<div id="full-option-list" className="indent">
<h1>TSConfig Reference</h1>
</div>
<nav id="sticky"><ul><li>...</li></ul></nav>
<div className="indent">
<div dangerouslySetInnerHTML={{ __html: post.html! }} />
</div>
</div>
</Layout>
)
}
export default TSConfigReferenceTemplateComponent
// The optional GraphQL query to get info for that page
export const pageQuery = graphql`
query TSConfigReferenceTemplate($path: String, $tsconfigMDPath: String!) {
sitePage(path: { eq: $path }) { id }
markdownRemark(fileAbsolutePath: {eq: $tsconfigMDPath} ) { html }
}
`I like this, the rest of the team don't need to learn React - just JSX. I still get the advantages in tooling because all of this is supported by TypeScript:
By not using any of the React One advantage of this has been that I can reliably run Would I recommend this technique to people making Gatsby websites, probably not, it's going against the grain (React is a really good tool) of how you're expected to use Gatsby. But the trade-off is worth it for me, and I spent some time thinking about that. SpeedI'm blown away by how fast Gatsby is for a user. The founder of Gatsby, Kyle Mathews gave a great talk in 2017 on the ways in which Gatsby is fast and here more recently, in rough:
His long term visions is to think of Gatsby as a compiler which takes a set of input source files, and will keep trying to make a static output which is faster and faster for users. Another great resource for understanding the mechanics about why Gatsby is fast is this talk by Nicolas Goutay at GOTO 2019. |
|
I took ^ and turned it into a well rounded blog post for the Gatsby blog: https://www.gatsbyjs.org/blog/2020-01-23-why-typescript-chose-gatsby/ |
This comment has been minimized.
This comment has been minimized.
|
Playground PluginsI admit, this wasn't on the roadmap. But, it was a weekend project, and it's worth talking about because it is my final big personal TODO item for the Playground. Developer tools are intrinsically interesting because their users have all the skills to contribute back to it. This is basically the crux of how the OSS world works. You get the lib/app/tool and the code, changes/improvements are culturally expected to end up going back to the main tool. This means everyone gets to benefit from the work of others. I came to JavaScript with a decade or so of native development experience, (thanks React Native!) and one of the most refreshing parts of that experience was the ability to own my tools and to really have the capability to fix any problems I saw in the ecosystem. This relied on two principles:
If this was the programming ecosystem you matured in, congrats - but it really isn't like that everywhere. Not being able to extend my text editor, or understand why a toolkit doesn't work the way you want is the majority of my programming career. One of the big struggles of running a successful open source project is how many people have the ability to contribute, and how often their needs just don't fit with a project's vision. This... is a shame. I wrote about this back in 2016 under the idea of "Defensive OSS":
I learned to build open source projects which have a small (ish) core, but are extensible in ways you can't predict:
Now, today, the TypeScript Playground is in that list. I can now give the right answer to folks asking "I'd love to be able to present in the Playground" which is "Built it yourself! Make a plugin." (or extend mine). Do I know what people will do with it? Nope. That's the good bit though. Making a Plugin Eco-systemI've done this a few times now, so here's what I think you need to start up a plugin eco-system:
Each one of these address funnel transitions as people get closer to building something cool, you know you've succeeded in building a good plugin infrastructure when people start making fun & frivolous plugins: Why? Because the barrier to entry is so low that you can try a throwaway idea. I guess I'm excited to see when we get So, for the Playground Plugins:
Then to dog-food plugins, the v3 version of the playground uses plugins itself for all of the sidebar tabs. This means anything I need to get the playground up and running is also available to any plugin developers. Plugin DownsidesI'd be amiss if I didn't at least talk about the downsides of a plugin ecosystem:
Playground PluginsNone of those are too much of a problem for the playground, I think. If it does, then I'll improve it in the future. In the meantime, I'd love to see some plugins from folks! |
ghost
mentioned this issue
Why is the new TypeScript website Internationalised?I grew up in an English speaking county, so I was never forced into a position to learn a second language. For the major of the world though, folks had to learn English on top of their own native tongue - either through exposure to culture or as a potential route for self-improvement. I've always had a chip on my shoulder about this. In-fact, I moved to Brazil for my first programming job explicitly to try learn a second language and I'm so thankful I got the chance to do that. The only blocking question I had for the interview was "Do we program in English?" - turns out everyone does. I spent the next decade and a half basically only writing software which works in English, and it's time to stop that. This year when I visited Brazil for BrazilJS, a talk by @sudowilliam (sorry, it's in Portuguese) really hammered the point home: By only providing documentation in English, only 3% of Brazilians could use the resources I was creating. Access to learning English is also effectively class based. If you want to help with social mobility then helping to lower barriers by translation is a great way to contribute. In Microsoft, everyone has a bi-annual set of goals/deliverables and luckily for me one of them has to relate to diversity and inclusion of some sort. You can read my self-review but I choose a stretch goal of getting the new site internationalized. Turns out doing it is significantly more complex than just saying "let's do it" - so, I've been treating internationalization as my personal side-project since wrapping up Flappy Royale and it's really starting to come together now. Internationalizing the TypeScript WebsiteWhen your tools are pretty low-level like Gatsby, internationalization isn't just a tick box you can hit and magically start producing sites which work logically across many languages. This website needs to be deploying as static HTML files for every language, and so we can't rely own runtime trickery. You need to think of how you are going to approach this from many different angles: Incremental Language AdoptionShould a translation be available publicly if not everything is available? IMO, yes. My technique has been to effectively treat a translation as a layer above English - so for every potential set of translatable documents, you can add only one to get the language URLs set up, and anything you haven't provided yet will have the English link provided. Given that we're relying on the OSS community to do translations, I don't think it's reasonable to put some kind of informal SLA on their time to block new parts of the website in English coming online. It can appear in English, and as someone finds the time, it can then be translated. URL schemesA very reasonable constraint we had was that cool URIs don't change - switching to the new website should not break old links. This meant that English gets to be a bit of a special case in the TypeScript website. For example:
This is not ideal. My initial goal was to have LinkingInternally there is a custom linking component which understands the patterns of internationalized URLs in this website. This means that I can write internal, English, URLs inside the website. Then depending on the current language of the page, it will resolve to either the localized page or an English one. This simplifies the React components considerably, it's kind of wild thinking that every internal anchor knows the URL for every page on the site - but the way gatsby is architected makes that pretty easy to do. Content GenerationThe TypeScript Website v2 is a monorepo, where I treat sections of documentation as unique packages. There are packages for:
Each of these has a The TypeScript website then uses these output folders to generate pages on the website. This two-step process is made easier by using watchman to automatically run the build scripts when markdown/ts/json files are changed in documentation. StringsBetween the options for Internationalizing React components, there wasn't an outstanding and obvious highlight. They all seemed good, but React Intl felt like it was the closest to the to the TC-39 standards - so I went with that. There are per-language sets of TypeScript objects which are simple key-value objects for the name of the copy and then the languages copy. The site uses TypeScript types to ensure every language has every string (again, they have English fallbacks if missing too) by using TypeScript's Uplifting TranslatorsI want to provide as many tools as possible for folks wanting to pitch in on translations. Their questions and blockers are one of my highest priorities because their work is unique and important. To give a space for collaboration, we create a channel in the TypeScript Community Discord, then an issue which shows how many files need translating. On a nightly basis, there is a GitHub Action which updates these issues. Then I added the ability for translators to use the CODEOWNERS feature of GitHub as an access route for merging their own PRs. This means they don't need to rely on one of the TypeScript team to merge PRs (if all of their PR changes are inside their code owners section) I'd like to provide a way to give attribution in the site too, I have some designs which handle this for individual documents (see the bottom) and I've still got to think about how it could be done for something more complex like the playground/tsconfig. Given that the translation effort is still a work in progress and has only really started in the last two weeks - there's been a lot of great work happening so far!
|
|
Not an essay (as it's half-way through the usual 2 weeks cycle) but I gave an internal talk on all the different parts of the TypeScript website for the team which is on YouTube. It's an hour long and I feel like I mostly covered everything at a high level.
It covers:
|
|
V2 Release CandidateI'm starting to wrap up the v2 site now, I'm pretty sure that all pages and urls are accounted for and everything I wanted in for launch. I think it's worth taking the time to look back at one of my first issues on the TypeScript repo as a team member: "What do you not like about the TypeScript Website and Documentation?" This issue defined the larger roadmap for my last 6 months as the main focus of my work in both upgrading the existing site infra, and preparing the infra for the next generation of the site. Fully Addressed
The new site has new navigation, and search is a core component to that! There's a non-technical doc on TS, but I've not found a place for it yet
The new playground is stunning, well documented and best of breed.
The tsconfig reference covers almost every compiler option with a comprehensive example.
The docs and handbook v1.1 changes cover all of these, I actually started moving some docs out of the advanced types page and into their own pages in the handbook. The v1.1ness of the Handbook brings over some of the opening v2 handbook pages and it restructures the handbook from a single "Handbook" (23 pages) to
Which should provide a bit more focus. Mostly Addressed
There's now a plugin API for the Playground, so anyone can fix these. I have a twoslash playground, but I've not investigated how feasible it would be to port fourslash to the web.
I've got the designs, I've just not built this bit yet. Dropped it for launch, but it should happen. Partially Addressed
The extensive Playground samples didn't exist before this issue, and they were built with very "everyday" code, e.g. using "APIResponse" types instead of "FooBar". They're not a full v2 handbook, but they're a much better than it was mid last year.
In v2 there's a better, and more expansive community page.
There's a lot of my personal notes in the TypeScript wiki now which includes some of these, but I'm finding myself really not liking the TypeScript wiki (because of it's internal implementation details, makes it hard to script and do automation) and want to minimize it's growth now.
I improved this a bit, but not enough, these are still the best resources:
I occasionally ship JSDoc improvements to the compiler, as I fix compiler bugs and hopefully when TS uses Modules it'll be feasible to use TSDoc to generate a page. There's tension in the TS team about the compiler API as a public API. I'd like to start moving docs on typescript into the compiler, and have a useful sandbox-like environment for exploring and learning.
There is active discussion about this.
There's the TypeScript Sandbox, which would share a cache - but I've not decided what this should look like and if it's not better to just send people to the playgorund instead.
I might try re-enable the PWA plugin for Gatsby, during active dev it was more pain than it was worth.
The v2s playground supporting mobile fully is blocked on monaco, which doesn't support mobile. Unaddressed
I'm still not sure if this is something I'd work on, too many projects and preferences.
Lots of recommendations for one off docs pages, which will get thought about in the future.
With the DT website domain going down, moving those docs into the website has a new urgency.
TypeScript had a lot of these types of repos when I arrived at the team, they were all a few years out of date and had tens of open PRs. I think this space is open for sandbox-y pages where we can show before/afters and let people explore how things work.
I think we could maybe provide a definitive "eslint with typescript" doc, but why compete with the official docs for typescript-eslint when we could link to it instead.
This is basically an infinite list, there is already a massive FAQ and more can be added to it over time. I'm sure there will be a bunch of bug reports when we switch to v2, but the foundations have felt pretty solid for a while now. After the flurry of bug fixes, I'll be switching over to bug fixes on the compiler for a while full time so that I can understand and document the compiler API better. I'd still expect updates to this issue because I still can think of a bunch of doc & infra updates which I'd like to see and talk about. Things I'll be thinking about in the meantime:
|
|
Hey folks, yeah, it's been way over a month. That said, this month has been more like 6 months of pre-2015 time, so I don't think anyone's judging. I ended up taking the time I use to write these updates to write up more official release notes for the blog. Given that it's during a TypeScript beta phase though, I usually tone down the amount of work I do on the site / docs to less than a day a week and just focus mainly on compiler bugs and some DefinitelyTyped stuff. Deployment#385 triggered a lot of "we really need to grok how all the azure side works". So, I set up a new staging website, which is easy to memorize www.staging-typescript.org which uses azure as a completely static site. I've been using that more than the InfraI shipped all the node modules which power the v2 site:
REPLPlayground Plugins have been getting some adoption:
And this morning I built one to show Code Flow Analysis nodes for your code. The plugin system paid off quite a lot because it made creating separate playground instances on the website easy. I started work on the Bug Workbench based off microsoft/TypeScript#35389, which was a feature that re-used the web infra modules to create single file bug reports which are almost the same as how we write tests in the compiler. This means we can request bug reports in this format, which then automated tooling can keep track of regressions and fixes
You can see here a link where you would expect the types to be the same, but you can see in "assertion found" that they are different. |
|
It's been a while ey. Progress has been slow, in part because I've been doing compiler / DT work full time and the sort of work which gets highlighted in here has been happening in my spare time since March. Thanks to COVID and the need to protest, I really don't have as much spare time to write software as I used to. Shipped npm modules
|
|
Surprise, it's a double post to make up for lost time. Since I mentioned the site was at RC, it has gone through some changes to try get the design past reviews, and an accessibility audit which is mostly on track now to be wrapped this weekend. With the goal of flipping the switch on the day of the 4.0 RC in early august. DocsI've started to focus on the content of the site given the plumbing is all there
Playground Changes
I have an idea how to build multi-file Playground, so there's a possibility that could happen in the future at some point. Follow this PR for updates. Misc
|
|
Subscriber bonus triple post. I spent some time building out a tool which is a bit like DefinitelyTyped but for TSConfig files. This comes from a few pains:
The last one is particularly tough for us, the 'recommended' settings is basically the result of 'tsc --init` which is something someone does once per project and never again used. No versioning, and when we update - it's not even in the release notes. TSConfig BasesThis solves the problem by using NPM in the same way Then you add the dep: npm install --save-dev @tsconfig/node10
yarn add --dev @tsconfig/node10Then use {
"extends": "@tsconfig/node12/tsconfig.json"
}That's it as a consumer. Now when you add compiler flags, they are always for your project and not for the runtime target: {
"extends": "@tsconfig/node12/tsconfig.json",
"compilerOptions": {
"strict": true
}
}RecommendedIt's still a WIP, but I'm trying to get internal consensus on a CurrentI've added base TSConfig files for these environments so far:
Built in DenoThis is a project built in Deno which, ironically enough, vendors out npm packages. It's mostly scripting and GitHub Actions but Deno makes a great scripting environment which I've used again since. |
Compiler-back Bug ReprosKeeping on top of compiler bugs is hard work, even just hitting watch on the TypeScript repo gets you many hundreds of emails a day from GitHub. Proving and validating a bug is easier now, thanks to the Playground improvements (supporting many TS versions, a lot more compiler flags etc), but it's still hard to keep on top of and Ryan C basically lives in an almost permanent state of triage in our issues. To give us the chance to improve our bug reports via automation, I wrote an RFC pitching that we can support letting people write compiler backed bug reports in issues/comments which we can use to know when a bug is fixed, and if it has regressed in the future. There are two parts to this system:
Making Good Bug ReportsThis is done by offering a web-app which gives you real-time information about the bug report you are making. Here's an example of the bug workbench which shows an inconsistency in the way we present a particular type ( microsoft/TypeScript#39262 )
The app uses the same twoslash syntax we use in code snippets inside the TypeScript website to highlight type information for a particular identifier. The workbench calls the results of these queries assertions, you can assert things like:
The site's goal is to teach you the syntax, and get you making repros quickly, so there's a lot of documentation in the app:
I've been using it to write code samples for the TypeScript v2 website as it's faster than using my text editor and the CLI. Verifying BugsOnce you've got your repro, it gets pasted into an issue or a comment on an issue. From here it gets picker up by the verification tools. This is a GitHub Action you can find at microsoft/TypeScript-Twoslash-Repro-Action and its job is to run nightly and provide useful information about the state of your compiler-backed bug report. You would write a repro like:
Then overnight the bot would post something like this:
Expanding the historical section, you can see that the verification step has also checked the repro against older builds:
So, it looks like the bug only existed for 3.8 and 3.9. This bug was looked at after I created the repro and is actually fixed now, so the nightly run updates the original comment and posts a new comment (so you get emails) telling you of the state changes and giving SHAs for the TypeScript releases between which the repro got different assertions so you can verify and 🎉 the fix.
This should probably be put into action next week on the TypeScript repo, I mainly need to polish up the markdown that the bot posts and add a few integration tests, but the idea is now sound. Shout out to @arcanis's Sherlock which highly influenced this project. |
|
Heh, in the opening comment for this thread, I mentioned that I had some TypeScript blog posts in the works back in November. They never made it to the product blog, it's possible one of them will still make it out at this point, but I don't want to lose the time nor documentation they bring. Last week, I moved some important TypeScript infra to the Microsoft org: Make Monaco BuildsThe playground is our online TypeScript editor-like environment. In the playground, you can make a sharable link We use the playground in debugging TypeScript issues, part of our issue template when someone files a bug report on Decoupling the Playground from TypeScript versionThe previous version of the Playground only supported one version of TypeScript: the latest production version. This isn't optimal, there are people who don't update to the latest version of TypeScript instantly and the TS team uses older versions of TypeScript to verify issues. Supporting multiple versions of TypeScript in a website is quite tricky, because the version of TypeScript is a well-hidden abstraction inside Monaco. Monaco is the editor component used in VS Code and the Playground and Monaco ships with a version of TypeScript in a plugin-ish way using a project called 'monaco-typescript'. To get different versions of TypeScript support, we need to make a build of monaco-typescript, then a build of monaco using that version. This idea came from typescript-play. Architectually I think of it like this, with a weak link between the playground/sandbox and TS versions via Monaco
This means if we want to own the different versions of TypeScript in the Playground, and support more versions than the Monaco team provides, we need to build the infrastructure to support it. Shipping the PlaygroundThe TypeScript project tends to move fast, and within a span of weeks the code in our We deploy a build of TypeScript every night, which you can find on the This is done via a GitHub Workflow which clones the repo, compiles TypeScript and ships that version to npm at 7am. Before we get started, some terminology. GitHub Actions is the name of the system, which has two separate In my opinion, this name collision is confusing. As Actions is both the name of the CI-like service, and an runnable task inside a workflow. GitHub have a glossary for all the Action terminology in their help site. To create nightly builds of the playground, I needed to create a workflow which is triggered daily, to do this I created a yml workflow file which lives inside a folder named name: Daily builds of monaco-typescript and monaco of TypeScript
# For testing
# on: push
on:
schedule:
- cron: "0 8 * * *"
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: "10.x"
registry-url: "https://registry.npmjs.org"
# Lets us use one-liner JSON manipulations on package.jsons
- run: "npm install -g json"
# For Azure uploads
- run: curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
- name: Setup Monaco TypeScript
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: node ./publish-monaco-ts.js next
- name: Setup Monaco Editor
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: node ./publish-monaco-editor.js next
- name: Upload to Azure
env:
AZURE_STORAGE_ACCOUNT: ${{ secrets.AZURE_STORAGE_ACCOUNT }}
AZURE_STORAGE_KEY: ${{ secrets.AZURE_STORAGE_KEY }}
run: node ./upload-assets-to-blob-storage.js nextThe first two steps are GitHub Actions which prepare the environment to make it set up for shipping a node module. The next steps run the scripts for deploying the node modules. Getting the environment for publishing to npm took me longer than I'd like to admit on this blog. Here's the key rule: Make sure These scripts take the nightly build of TypeScript, and then wrap it with the essential web infrastructure to be
Actions via WebhooksNightly builds are cool, but do you know what's cooler? Pull Request builds. Work on TypeScript features can turn into very long running pull requests. One of the techniques we've used to help people test if a change improves their workflow is by making npm releases from these pull requests. We have a bot which listens to GitHub comments from the core team in Pull Requests, one of us needs to say:
Saying that in a comment will trigger a job on Azure DevOps which builds and uploads a gzipped tarball. People can use people can install TypeScript from this tarball via npm. I wanted to extend this process to start building a copy of the website with that version also. This turned out to be a one-line addition to the DevOps build, and used a feature of GitHub Actions which lets you send trigger webhooks to a GitHub repo. These are called repository dispatches, This string can be grabbed from a JSON file which you can find at To hook that up, I added a new yml workflow file in the repo which is triggered when a repository dispatch event name: Build a version of TypeScript on request
on: repository_dispatch
# This workflow is triggered from an API call where XXX is your token
# and YYY is the PR that it should look at.
# curl https://api.github.com/repos/microsoft/typescript-make-monaco-builds/dispatches \
# -XPOST \
# -H 'Content-Type: application/json' \
# -H 'Accept: application/vnd.github.everest-preview+json' \
# -H "Authorization: token XXX" \
# --data-binary '{ "event_type": "YYY" }'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: "10.x"
registry-url: "https://registry.npmjs.org"
# Lets us use one-liner JSON manipulations on package.jsons
- run: "npm install -g json"
- name: Publish PR Build
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
yarn install
PULL_REQUEST_NUMBER=$(json -f $GITHUB_EVENT_PATH action)
echo "Publishing Monaco based on $PULL_REQUEST_NUMBER"
...snipRelease DeploysFinally, we need to be able to create release builds. These are triggered by creating a git tag for the version Next StepsThat's it for GitHub Actions in building and deploying custom builds of the TypeScript playground. GitHub Actions takes a lot of the best practices in continuous integration services like in-repo config, shared actions and they provide a fast and consistent way to build automation into your daily work. Building off the webhook system is a powerful abstraction, and prior to GitHub Actions I spent about 2 years building a similar system called Peril. I feel good about where GitHub Actions is and where it will go. |
Playground Collab v2Hey folks, this is as good a place to announce as anywhere else, but I've shipped a new build of my multiplayer version of the Playground. The main goal was gross simplification. To be frank, I'm not building something as good as live share in my spare time, so I needed to figure out how to cut enough features that I could build & maintain it myself. My goal was to have something which works for 2+ people in a video/voice chat who can share a single Playground. E.g. our design meetings. This actually means that I can cut features because you have a way to communicate, and it's not on this tool to do that:
Now that it's actually feasible for me to build it. I set up a TypeScript owned SignalR instance which I essentially treat as a client-to-client multicast server. In v1 you used to log in via Oauth, in v2 there's no authentication - you just set a name, room and hit connect.
I was generally embarrassed at how janky the authentication was. Auth is like the top of the user-funnel, and you want to just look at some code, it's not gonna be stored or too private if it fits into a playground. It didn't need to be there. Now it's trivial to set up and collaborate:
How it works is every 3 seconds all connect clients send a message to each other which:
Each client then keeps a map of the usernames to their last messages and updates the UI which is roughly:
You can try it by clicking on this link: https://www.staging-typescript.org/play?install-plugin=playground-collaborate |
|
Hi folks - got a 3 reasonably interesting updates from the weekend, first up, the site now has a color theme switcher for light/dark based on the work of @dandelionadia. There's no flash when you reload (usually noticeable when the choice is the opposite to your OS preference) and the design has space for allowing you to choose the code samples dark/light-ness in the future.
Next up is that the final parts of the internationalization efforts is done. This is the work for getting all of the markdown documentation in order, which means per-language sidebars and individual pages. You can see here a lovely l33t-sp33k port of the Basic Types page in the docs.
This wraps up the final pieces of internationalization, with the site almost launched then hopefully we can get more languages fleshed out! Finally, a fun one is that the compiler-backed code samples infra I talked about earlier is now up-and-running on microsoft/TypeScript and I built out a bot for letting folks know when a repro has been requested or received on an issue making it a good way for people to test their 'is this a bug' skills
|
This comment has been minimized.
This comment has been minimized.
|
It's not strictly TypeScript Web Infra, but I've been working with the Svelte team for the last few months on getting TypeScript support and tooling to feel native. It's pretty cool work, you can read more here: https://svelte.dev/blog/svelte-and-typescript |
DeploymentI'm a pretty conservative programmer with respect to dependencies, especially in the JS ecosystem. I come from a native Mac/iOS background and basically everyone in that ecosystem is cautious with their approaches to dependencies and views them mostly as a necessary burden. For example, one of the threshold moments in my career was the rejection of of advice from an Apple developer rep when they said I should remove all my dependencies in our iOS app and write all the code myself ranging from image rendering to networking. I disagree now, and I disagreed then. Shared concerns can be shared with others. Understanding this is why I worked (and still occasionally help out) on the iOS dependency manager CocoaPods for almost a decade. The JS eco-system won the race to the bottom for ease of sharing dependencies (did you know you don't even need a package.json for an npm module?) This has it's advantages for sure, but I think the trade-off for this scale is that you you lose some of the useful human elements in the automation to make it work. ( There's a good book on where the balance in humans and machines can lay, Machines of Loving Grace which details the battles between AI and IA where AI effectively won at our cultural loss. ) The symptoms of an over reliance on automation tends to be a lot of small dependencies with hard to grok automated changelogs, a README covering a subset of the API, automatic deploys directly from master and often an embarrassingly large dependency tree ( the TS v2 website has an astounding 1,778 folders in node_modules despite my attempts to keep it down (but hey the pages loads in 0.x seconds and has close to no deps for users ) ) I try to avoid some of these pitfalls when working in the JavaScript ecosystem, Danger JS (36 resolved deps) is a good example of a tool which can be used to provide more of a human touch to a project (by providing primitives for enforcing useful changelog entries, comparing READMEs to exposed APIs etc) Over the years, I've accrued quite a few different ways to handle npm module deployment, and yesterday after making yet another new way to handle deploys I figured it was worth listing them and when they are a good fit. PR Driven DeploysWhen basically all your source code contributors have write access (a.k.a work projects) then you can declare at the PR level whether merging this PR should trigger a deploy or not. The versioning system for patch, minor or major would be derived from labels. I started building this and then was pleased to discover that @hipstersmoothie had already got really far in the process via auto and opted to write up how it worked instead. Nightly Deploys if NeededFor projects which sometimes see a burst of activity, then periods of rest. For example in the TSConfig bases repo which checks the git history and determines if there were changes before deploying: - run: |
# Have commits been added in the last day?
if [[ $(git log --pretty=format: --name-only --since="1 days ago") ]]; then
git log --pretty=format: --name-only --since="1 days ago" bases | deno run --allow-read --allow-run --allow-env scripts/deploy-npm-packages.ts
fi
env:
NODE_AUTH_TOKEN: ${{ secrets.NODE_AUTH_TOKEN }}
name: 'Deploy built packages to NPM'Nightly deploy trains offer the chance to make sure everything is right before actually deploying. E.g. if contributors didn't add CHANGELOG entries, or if a human needs to re-run automation ahead of time. This commit log checking style only works well if you are working with a single module, so for monorepos I have a GitHub Action monorepo-deploy-nightly which is a bit more nuanced. We use this for nightly builds in the Svelte Language Tools repo and I'll switch the TypeScript website deploys to this when v2 is on production. Git Tag Based DeploysSome projects require a distinction between production and staging. Deployments to staging can be done on master merges or nightly that's pretty reasonable, then a production deploy can happen when you push a tag. This is how the Svelte VS Code extension production deploys work. Comment Based DeploysThis is, in part, how TypeScript deploys work - here's the start of the 4.0 infra. A bot/GH action can be registered for issue comments, verify the author has write access and then start doing the work. You can use the search term chat-ops to find some interesting ideas around this. Version Check Master DeploysThis is useful when you have a rarely edited but well used dependency, I added this to dtslint last night: name: Deploy to npm
on:
push:
branches:
- main
- master
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
registry-url: "https://registry.npmjs.org"
# Ensure everything is set up right
- run: "npm install"
- run: "npm test"
- uses: orta/npm-should-deploy-action@main
id: check
- run: "npm publish"
if: ${{ steps.check.outputs.deploy == 'true' }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}I built out a GitHub Action which compares the package.json version against npm's and that has a build output of whether to deploy or not. This is a nice abstraction because it's secure, note that the npm auth token is in a completely separate step so even if that repo is compromised then the worst it can do to dtslint is tell it to deploy at the wrong time which would fail. For monorepos which only deploy to npm, like the TypeScript Website v2 I use pleb to deploy only when the versions have changed. Manual DeploysFor Danger in JS and Ruby, I still deploy manually from my computer. These are more unique projects and that's because they are built on the Moya Community Continuity Guidelines - it's a set of cultural rules to enforce the long term life for important shared dependencies. Basically, if you get a PR merged you are automatically invited into the GitHub org with write access. We use the distinction of having the right to publish the final release to npm/rubygems/maven as being the distinction of core contributor vs someone who has the option to help out. That means if I stop using ruby Danger (I did) and or disappear from the internet (possible, but unlikely) then others (~260) have the tools to step up and maintain the repo without me doing anything today. |
Watch Me if You CanPopular web frameworks made in the last few years have a mode where you can start a local server and see changes as you press save. These are commonly known as watch modes. TypeScript has one! Gatsby, the JS framework which the TypeScript v2 website is built on, is no stranger to a local build server. Gatsby's dev mode does a great job of keeping track of how changes should propagate through your dev build for example it understands that changing a markdown file should affect a particular page. It's fast and solid, I'm willing to trade some ahead-of-time work while it's setting up to get fast and accurate changes on save. Note: keep your eyes on jspm, vite and Snowpack as a preview of what your JS tools will look like in 2-3 years. When things get drastically simpler due to ES Modules. If the v2 site was a single project repo, then the Gatsby dev server would be enough. However, it's not. It's a monorepo of 12 packages with some being:
Some of these packages have watch modes, but to be fair, it's a bit of a hassle to run many watch mode servers as you're iterating - so what do you do? My solution came from the pattern used in React Native under the hood, and that is the tool Watchman. Watchman is a native C++ library which uses your OS file change notification systems and provide a high-ish level API for other languages, JavaScript included. Running I made some changes to the structure of the monorepo in order to make a watcher for the whole repo, and that was to standardize on using With that done, I created a watchman script which monitors for changes in markdown, TypeScript and JSON files and figures out which package the changes come from. Then it runs Here's the file, but here's the general gist: // A script which uses Facebook's watchman to run `yarn build` in different modules
// in a standard monorepo.
const watchman = require('fb-watchman')
const client = new watchman.Client({})
const chalk = require('chalk').default
const projectForFile = (file) => {
// skip triggering the watcher from derived files, Gatsby grabs those
if (file.name.includes('/dist/') || file.name.includes('/out/')) return
if (file.name.includes('/typescriptlang-org/')) return
if (file.name.startsWith('packages/')) {
return file.name.split('/')[1]
}
}
let upcomingCommand = null
let currentProcess = null
// All this is basically a bunch of boilerplate code to set up a watchman
// for the project which looks only at .ts and .md files in the repo.
// Startup watchman
client.command(['watch-project', process.cwd()], function (error, resp) {
// https://facebook.github.io/watchman/docs/cmd/subscribe.html
client.command(
[
'subscribe', root, 'Monorepo Builder',
{
expression: ['anyof', ['match', '*.ts'], ['match', '*.md'], ['match', '*.tsx'], ['match', '*.json']],
relative_root: path_prefix,
fields: ['name', 'exists', 'type'],
},
],
// error handling
)
client.on('subscription', function (resp) {
// NOOP for changes large amounts of files (`yarn install`s)
if (resp.files.length > 10) return
// Get the changed files, convert it into an array of changed packages:
const projectsToBuild = resp.files.map(projectForFile).filter(Boolean)
const uniqueProjects = Array.from(new Set(projectsToBuild))
// For simplicity, I don't wanna handle multiple processes
const commandToRun = uniqueProjects.map((project) => {
const packageJSONPath = join('packages', project, 'package.json')
if (!existsSync(packageJSONPath)) return
// yarn workspaces uses the npm name, not the folder name
const packageJSON = JSON.parse(readFileSync(packageJSONPath, 'utf8'))
if (!packageJSON.scripts || !packageJSON.scripts.build) return
const buildCommand = `workspace ${packageJSON.name} run build`
return buildCommand
})
if (commandToRun[0]) {
if (currentProcess) {
upcomingCommand = commandToRun[0]
} else {
runCommand(commandToRun[0])
}
}
})
})
const runCommand = (argString) => {
// run the command via `execSync` etc
}This means that as you work in any part of the monorepo with the dev server running, pressing save in any source file will trigger running |
|
Alright, it's happening - I ran https://www.typescriptlang.org/Over the year, this project has bounced between the thing I work on full time and a side-project I keep running on TypeScript time. That said, the TypeScript website has been my only free time project since Flappy Royale (with the svelte TS tooling being a necessary exception) and there's a part of me that really doesn't know what to do with that time now that getting the site done is less pressing. I'm planning on doing a live Q&A today at 5:30 EST on the TypeScript Website: https://www.twitch.tv/ortatherox/schedule In the mean time, I'd like to give some thanks:
🍾 🚢 |
|
Small update on a cool project I built while I took a week off. It was live-streamed on twitch, video here. Code Owner Self MergeThis is a GitHub Action which can work on any repo to give people merge access on PRs only when files which they have access to change. It uses the existing CODEOWNERS infrastructure for defining the access control. We've had a bunch of PRs self-merged into the TypeScript website now, so I'll walk you through how it's set up on the website. From a Community Contributor's PerspectiveSomeone submits a PR, this PR only edits a single translation file in Portuguese
This triggers the usual CLA bot, and the Code Owner Self Merge Github Action also replies: it lets @khaosdoctor, @danilofuchs, and me know that this PR can be merged by any of us saying "LGTM." The translation is reviewed, and then @danilofuchs says that the translation looks good and can be merged
This is a bit of a simplification ( see #926 for more context ) but this is how it looks like as someone contributing. Under the HoodA GitHub Action can run against all sorts of webhooks, for this action we care about: PRs, issue comments and PR reviews. The workflow looks like this: name: Codeowners merging
on:
pull_request_target: { types: opened }
issue_comment: { types: created }
pull_request_review: { types: submitted }
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Run Codeowners merge check
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
uses: orta/code-owner-self-merge@masterThis means run the action whenever any of those events are triggered. The first one is when a PR is made. The action looks up the CODEOWNERs file in the standard places, for this repo, it's at # Collaborators for Portuguese Translation of the Website
packages/playground-examples/copy/pt/**/*.md @khaosdoctor @danilofuchs @orta
packages/playground-examples/copy/pt/**/*.ts @khaosdoctor @danilofuchs @orta
packages/tsconfig-reference/copy/pt/**/*.md @khaosdoctor @danilofuchs @orta
packages/typescriptlang-org/src/copy/pt/**/*.ts @khaosdoctor @danilofuchs @orta
packages/typescriptlang-org/src/copy/pt.ts @khaosdoctor @danilofuchs @orta
packages/documentation/copy/pt/**/*.ts @khaosdoctor @danilofuchs @ortaThese are a set of globs to different packages, and all the places where Portuguese translation occurs. It might remind you of how a The action determines the changed files via the GitHub API, for this PR it finds: Which fits the glob: The action determines that Later on, in either a review or a comment - if one of us three owners say "LGTM" then the action will merge the PR. I like systems like this. It offers a privilege access to those who are doing useful work, and makes them not feel blocked by having to rely on someone on the TS team to merge their PRs. This is especially useful for translation PRs, I can't review those! These contributors are donating their time, and our tooling can support it by getting us out of the way while knowing that this can't break the TS website. |
Twoslash 1.0 ReleaseI think twoslash is now at a point where you probably don't need to read the source code to understand it, which is as good an indication as any to call it a 1.0. It'll get shipped when #965 is merged. I've taken the last few days to tighten the errors, add the features which are important for projects which aren't the TS website and improved some of the infra surrounding it. Here's the state of the art:
|
Sample SEOA few weeks ago I fixed an issue which meant that each playground sample had a corresponding
This means that we we can really target providing comprehensive answers to really specific questions, and get them in a place where people can find them. I've not really talked about this to anyone, so it seems like a reasonable topic to write-up. Why?One of the problems with documenting any language is that it kinda covers a lot, and that sometimes the answer can be so deeply technical that most people don't need that exact answer. We often need to provide the same content at different levels of technical expertise ( e.g. look at the 4 getting started docs before the handbook ) and one thing the TS site was missing was small focused REPL-like examples which can give you an overview of the features in a way that encourages playing around. This gave me the chance to build out a new section of docs in a way which I could use to test the waters for how the TS team thinks and reviews docs changes to the site and I tried to cover a lot of topics. I based the structure originally from a glossary I had been keeping of TypeScript terminology: Each file is a This is a good system, but it had discoverability problems as they were only available from inside the Playground. My first solution for that was to add it to the footer of every page of the site under "Code Samples". That helps, but it didn't really address what I was looking for, which was being able to search for them externally. To figure that out, I added some code to the site's boot up sequence which creates a page per example and basically switches the comments with markdown, and moves the code into a code block: Converting this:
Into this:
Then each page checks if you're a search engine or not, and forwards humans to the Playground: const Play = (props: Props) => {
const i = createInternational<typeof headCopy>(useIntl())
useEffect(() => {
// Keep this page around so it is indexed on search engines
const isBot = /bot|google|baidu|bing|msn|duckduckbot|teoma|slurp|yandex/i.test(navigator.userAgent)
if (!isBot) {
// @ts-ignore - this is allowed in the DOM
document.location = withPrefix(props.pageContext.redirectHref)
}
}, [])This means that as we document features using Playground examples, they will be easily accessible for anyone with a search engine! This works across languages too, but I think we probably need a way to handle translating the title at some point.
|
Localized LocalizationsThere are about 700 pages on the TS website today, with over 1000 sections localized into 9 languages by the community. This rocks. I've mentioned it a bunch of times in this blog-issue-thing, but letting people learn in their own language is real important to me because it lowers the barriers of entry to TypeScript considerably. We're getting to a point where the amount of PR traffic for the localizations is starting to dwarf PRs to the website, and that can make it hard to stay on top of the repo for anyone other than me. So, at the start of 2021, I pitched that I was planning on splitting the localizations out, and would make the tooling generic and available for others. This weekend I'll flip the switch to move the localizations out. Ah, but what's in it for you dear constant reader? Well, if you are thinking of having a large corpus of localized markdown files, you can re-use:
All of these tools lives under a new GitHub org OSS-Docs-Tools (with a logo designed by my wife, Danger) and are usable out of the box today. There are some trivial example repos in the org, but the real production case is https://github.com/microsoft/TypeScript-Website-Localizations/ |
giphy-2.mp4Trust, but verifyI realized after reading @nayafia's book "Working in Public" ( this is a good primer for those who haven't read it ) that I have been working on the problem of scaling human issues inside OSS communities for about 7 years now. As originally, I hit these problems working on the iOS dependency manager CocoaPods back in 2014. Just like with TypeScript, for any non-trivial OSS project there's nearly always a small team of core contributors and a lot of issue feedback and drive-by PRs which can get overwhelming, even when working full time on OSS. In this case a side project (localizations) of a side project (website) for TypeScript can still generate a lot of traffic. There were 10 localization PRs last week and as someone who can speak English (and barely get by with Portuguese) - I cannot provide constructive feedback on a Japanese translation PR. So, how can I handle the growing needs of contributors without burning myself out trying to really get to grips with each language? My technique has been through rigorous application of automation which helps maintainers put more trust in contributors. I've just wrapped up moving the localization infra to a separate repo, and would like to cover the security model of how I let it more-or-less run without the TypeScript team's direct involvement. Just to get you up-to-date, the TypeScript website is a static site which reads markdown documents, TSX files and info from the TypeScript compiler to generate a set of HTML files using Gatsby. Localization is handled by file shadowing: e.g. a lookup for a markdown document will first look up for the locale specific version Narrowing the Safe ChangesAt a pull request level, we can determine how "safe" a change is by looking at what files have changed. Does this change affect files which aren't localization related (e.g. the This idea is a simplification of the far more complex DefinitelyTyped access rights system. In the localization repo, this is handled by re-using the system of code-owners which GitHub introduced back in 2017. The GitHub implementation is for reviewers, and so people without write-access to the repo cannot be assigned to review, there is a GitHub Action OSS-Docs-Tools/code-owner-self-merge which turns it into an authentication system for external contributors. How it works is that if someone included in the codeowners owns all files in a Pull Request, then they have the ability to request that it get merged via the GitHub Action: # Collaborators for Portuguese Translation of the Website
docs/playground/pt @khaosdoctor @danilofuchs [translation] [pt]
docs/tsconfig/pt/**/*.md @khaosdoctor @danilofuchs [translation] [pt]
docs/typescriptlang/pt/**/*.ts @khaosdoctor @danilofuchs [translation] [pt]
docs/documentation/pt/**/*.md @khaosdoctor @danilofuchs [translation] [pt]Here's some example of files in a PR, and whether @khaosdoctor and @danilofuchs can merge:
This means that the maintainers can know that people who are just working on the content in the repo are working on their own section, and not touching infrastructure files (or disabling the CI etc.) Non-breaking ChangesIf the localization repo was only simple markdown documents, maybe the above would be enough. However, the markdown documents have specific metadata, and compiler-backed code samples which can throw. Even worse, the inline localizations (like the text for the navigation) are actually evaluated code. Hrm. So, the repo has a pretty strict linter to help give fast feedback to contributors. For the markdown documents, it runs the same library (twoslash) and will validate the YML metadata. For the code, the linter uses the TypeScript API to validate the export const navCopy = {
skip_to_content: "Lewati ke konten",
nav_documentation: "Dokumentasi",
nav_documentation_short: "Dokumentasi",
}This file is validated to always be a single export which is a const, that creates an object with static strings. Template strings are also allowed, but not ones which use The other sort of files looks like: import { defineMessages } from "react-intl"
import { playCopy } from "./playground"
import { Copy, messages as englishMessages } from "../en/en"
export const lang: Copy = defineMessages({
...englishMessages,
...playCopy,
})Every import is validated, there's only allowed to be the one export statement which has to then be the function These linter rules use the TypeScript AST, so they allow for flexibility in the names and whitespacing etc - but don't allow for the way the code works to change. This linter isn't some fancy re-usable CLI tool, it's ~300 lines of JavaScript. All of these linter checks run on CI, and the GitHub Action won't allow people to merge PRs which are not green. This means we can trust that changes to the evaluated code don't cause un-expected behavior in the the TypeScript website. Easy IntrospectionFinally, there's one more automation point which is that a comment is added to any translation PR which translates it to English in the comments. This is done via Danger and an Azure API (see a PR here) - it's a nice way for us to have a check over contentious localization changes and offer some sort of insight. GitHub CheckedWith all of these validations in place, it's feasible to have a secondary authentication system in a repo which allows for people without write access to maintain their own sections of the codebase without breaking the upstream project which depends on it. Making life easier for people who want to translate, and simpler for maintainers who can't provide all of the useful feedback they'd need. |
How I built the Playground Plugins infraI gave a talk at TSConf 2020 on Playground Plugins which generated quite a few questions about how this was built technically. I think there are mainly two parts which are worth discussing, how code connects and how to think about the dev experience. Running the CodeTo connect the JavaScript files in the browser, you need a loader of some form. In my case, I was already using the vscode-loader for Monaco (the code text editor). This uses AMD modules, here you can see how TS supports it. I wouldn't be too surprised if this could be done entirely in ESM today, but this was built before ESM in browsers were stable. The vscode-loader setup which occurs way before any plugin code runs: // @ts-ignore
const re: any = global.require
re.config({
paths: {
vs: urlForMonaco,
"typescript-sandbox": withPrefix('/js/sandbox'),
"typescript-playground": withPrefix('/js/playground'),
"unpkg": "https://unpkg.com/",
"local": "http://localhost:5000"
},
ignoreDuplicateModules: ["vs/editor/editor.main"],
catchError: true,
onError: function (err) { ... }
});This lets you require a url like const downloadPlugin = (plugin: string, autoEnable: boolean) => {
try {
// @ts-ignore
const re = window.require
re([`unpkg/${plugin}@latest/dist/index`], (devPlugin: PlaygroundPlugin) => {
activateExternalPlugin(devPlugin, autoEnable)
})
} catch (error) {
console.error("Problem loading up the plugin:", plugin)
console.error(error)
}
}From here we allow for factory plugins instead, and register the plugin if (typeof plugin === "function") {
const utils = createUtils(sandbox, react)
readyPlugin = plugin(utils)
} else {
readyPlugin = plugin
}
// ...
playground.registerPlugin(readyPlugin)Check whether it wants to be at the front, and if so, activate it: // Auto-select the dev plugin
const pluginWantsFront = readyPlugin.shouldBeSelected && readyPlugin.shouldBeSelected()
if (pluginWantsFront || autoActivate) {
// Auto-select the dev plugin
activatePlugin(readyPlugin, getCurrentPlugin(), sandbox, tabBar, container)
}Plugins are kept inside an array in the Playground which sets the order for the tabs. There are a set of built-in plugins which provide the JS, .d.ts, Logs etc which are passed in at launch. By being configuarble we can support playgrounds with different default plugins like the bug-workbench. New user-defined plugins are push'd into that array, and are controlled with the same APIs as the built-in infra. The list of npm packages you've requested gets stored in localStorage and is pulled out on a reload and grabbed on launch. Development ExperienceToday, no-one is working commercially to make a plugin, so it should be set up for quick feedback and easy hacks to let people explore the space. SetupThe constraints from above are that to make a plugin you need to:
That first one is a big blocker for a dev environment, so I found what I believe is a security flaw in Chrome - any web page can connect to So, I made a second constraint: you can opt-in to auto-connect to a dev plugin at // Dev mode plugin
if (config.supportCustomPlugins && allowConnectingToLocalhost()) {
window.exports = {}
console.log("Connecting to dev plugin")
try {
// @ts-ignore
const re = window.require
re(["local/index"], (devPlugin: any) => {
console.log("Set up dev plugin from localhost:5000")
try {
activateExternalPlugin(devPlugin, true)
} catch (error) {
console.error(error)
setTimeout(() => {
ui.flashInfo("Error: Could not load dev plugin from localhost:5000")
}, 700)
}
})
} catch (error) {
console.error("Problem loading up the dev plugin")
console.error(error)
}
}This means you can work inside the production version of the Playground with your dev build of the plugin. Then when you're happy, you can ship it to npm. With constraints like the above, you really need to provide a bootstrap environment. To help people get set up quickly, you could use a Yeoman generator - I forked initit which also included Playground .d.ts files and shipped it as its own npm package: yarn create typescript-playground-plugin playground-my-pluginThis sets the dev up with a rollup environment for working with both a watch mode and a way to bundle for production. The default setup was enough for me to build a few plugins, and it includes examples for more complex rollup config. APIThere are two things to think about:
I built the Playground plugin's API inspired by how I would build a tab controller in iOS native: /** The interface of all sidebar plugins */
export interface PlaygroundPlugin {
/** Not public facing, but used by the playground to uniquely identify plugins */
id: string
/** To show in the tabs */
displayName: string
/** Should this plugin be selected when the plugin is first loaded? Lets you check for query vars etc to load a particular plugin */
shouldBeSelected?: () => boolean
/** Before we show the tab, use this to set up your HTML - it will all be removed by the playground when someone navigates off the tab */
willMount?: (sandbox: Sandbox, container: HTMLDivElement) => void
/** After we show the tab */
didMount?: (sandbox: Sandbox, container: HTMLDivElement) => void
/** Model changes while this plugin is actively selected */
modelChanged?: (sandbox: Sandbox, model: import("monaco-editor").editor.ITextModel, container: HTMLDivElement) => void
/** Delayed model changes while this plugin is actively selected, useful when you are working with the TS API because it won't run on every keypress */
modelChangedDebounce?: (
sandbox: Sandbox,
model: import("monaco-editor").editor.ITextModel,
container: HTMLDivElement
) => void
/** Before we remove the tab */
willUnmount?: (sandbox: Sandbox, container: HTMLDivElement) => void
/** After we remove the tab */
didUnmount?: (sandbox: Sandbox, container: HTMLDivElement) => void
/** An object you can use to keep data around in the scope of your plugin object */
data?: any
}Basically, a set of delegate hooks before/after when something happens, and an easy way to handle text changes in the editor. It's enough for everything I've built. The playground is effectively a "vendor" in this case, so what APIs I expose are basically about making many things easy, and hard things possible. I created a Playground design system. The design system gives folks the chance to not try to copy the playground aesthetics (and to support mobile/dark mode/custom fonts etc) and provides a simpler, high level DOM API. I intentionally didn't offer a React/Svelte/Vue plugin because I don't use these in the Playground already, and within a few weeks they popped up from the community. Because the site is React, we worked to expose the website's React to a playground plugin too. This means all three add 0kb to the page size to use. Then I have a lot of high level APIs which are meant to abstract over the internals of the monaco editor, so that you can think in "TypeScript"-level abstractions: return {
/** The same config you passed in */
config,
/** A list of TypeScript versions you can use with the TypeScript sandbox */
supportedVersions,
/** The monaco editor instance */
editor,
/** Either "typescript" or "javascript" depending on your config */
language,
/** The outer monaco module, the result of require("monaco-editor") */
monaco,
/** Gets a monaco-typescript worker, this will give you access to a language server. Note: prefer this for language server work because it happens on a webworker . */
getWorkerProcess,
/** A copy of require("@typescript/vfs") this can be used to quickly set up an in-memory compiler runs for ASTs, or to get complex language server results (anything above has to be serialized when passed)*/
tsvfs,
/** Get all the different emitted files after TypeScript is run */
getEmitResult,
/** Gets just the JavaScript for your sandbox, will transpile if in TS only */
getRunnableJS,
/** Gets the DTS output of the main code in the editor */
getDTSForCode,
/** The monaco-editor dom node, used for showing/hiding the editor */
getDomNode,
/** The model is an object which monaco uses to keep track of text in the editor. Use this to directly modify the text in the editor */
getModel,
/** Gets the text of the main model, which is the text in the editor */
getText,
/** Shortcut for setting the model's text content which would update the editor */
setText,
/** Gets the AST of the current text in monaco - uses `createTSProgram`, so the performance caveat applies there too */
getAST,
// ...
```'
I exclusively use these APIs in all the built-in plugins, and only really special-case the Logs/Errors plugins because they are connected to a lot more parts of the playground. That dog-fooding means they're pretty solid for plugin authors. |
|
Writing Puzzles for DevelopersI've just wrapped up the second series of How I Try To Structure PuzzlesThere's this perfect 5 minute YouTube video from Game Maker's Toolkit on the philosophy which I have tried to apply when thinking about each puzzle: Super Mario 3D World's 4 Step Level Design. Koichi Hayashida is the co-director of Super Mario 3D World, and roughly his puzzle philosophy is:
While I've not succeeded in hitting all of these points in every puzzle, that is the goal for each challenge. In the context of Writing ProcessCreating each of the 40 puzzles is a multi-step process:
In real-world times it comes to about 10 hours each days, and are super mentally taxing - so I usually take some time off the next week. DifficultyI learned quite early in my career that making a hard opponent is easy but making a fair one is hard. With these challenges it can be quite hard to figure out how difficult a concept is to learn (because I've already learned it) and in part that's why there's multiple peer review steps. The tools I use to modify the difficulty of a puzzle are:
I don't expect someone to figure out syntax from first principles, so it's usually a good assumption that if someone is experienced enough to know these puzzles exist - they know that they can search "[term] typescript" in DuckDuckGo to find something related. I try to make sure there's always at least one clue which can be searchable in every puzzle. 2021 PreparationUnlike 2020, which we were roughly guessing the format, we knew what 2021's version of For |
Direct Support for Technical CommunitiesWhen I started trying to scope out what the community side of the TypeScript web presence could look like, I thought about what sort of support paths are available to people using the language.
I think the Gitter came from the original open-source project and then was inherited by the TypeScript team with the project. We have someone on the TS team on rotation, but not everyone would use be available in the Gitter chat. So, if you were looking for help on a TypeScript problem where do you go? Stack Overflow is a good bet but there's a lot of people who don't post questions on Stack Overflow (myself included.) That left the Gitter for one-on-one questions, but Gitter's future is still a bit debatable as post-acquisition by GitLab the codebase has been on maintenance only mode and you only get one channel. We'd struggle to handle the size of the TS community in one channel. The other angle I was interested in was to consolidate the random chats for the different TypeScript meetups/conferences, remember this was the before-times. The 2019 meetup / conference scene was super active, and making their lives easier was high on my TODO because it was one of the ways in which I grew as an engineer and I wanted others to have that chance. So, we wanted a system like Slack or Discord. GitHub Discussions didn't exist at this time, and I think is worth considering about today. I think GitHub Discussions would make a good localized Stack Overflow replacement but will probably not be a place to hang out. GitHub's a bit too formal for that. Thinking about Slack vs Discord in 2019, the writing was largely on the wall for developer communities on Slack, it was something people put quite a lot of effort into making it work, but Slack wasn't too interested in communities and it showed. Having seen a few teams go from Slack to Spectrum and then Discord. I figured we should just jump straight to Discord. A naive approach would have been to click 'Create new Discord', but I think there's a tension there. Outside of myself, I couldn't (and wouldn't) guarantee the time of the TypeScript compiler team to actually be there to bootstrap it, moderate it and scale it. Instead, I looked at existing Discords for TypeScript and opted to talk to the admins for the "TypeScript Community Discord" which had 200-300 members which had shown up in Reddit once or twice. I effectively offered a trade: I can spend work time helping you to scale the community and start directing folks to this Discord, in exchange we'll need to set up things like Codes of Conducts, start moderating more strictly and give at least one person on the TS team moderator access. I will be here to offer a hand, but it's still your community to run. They were happy with these trade-off, and so we did a few things:
Over the last two years we invested in a few places: Help ChannelsWhen I joined the Discord, there were two separate help channels. Once we hit ~10k members this was not scaling. Discord did not support threading at the time, and so we modified the bot into a clever and nuanced system which can handle many help simultaneous requests by cycling a set of channels between being public or archived (for example there's 15 active help channels as I write this, and 18 archived. Which means we've peaked at 33-2 concurrent help channels. ) Given how fundamental the help system is to the discord, it is in continual evolution - there's been some interesting proposals to move it to threads and there's a generic bot which offers this for anyone here. Related CommunitiesOriginally we migrated in meetups to the discord, but over time as meetups stalled from the lockdowns, so did those channels. Instead we found that quite organically it started to make sense to host a channel for projects which were heavily reliant on TypeScript. For example TypeDoc and ts-node are there, as well as some more generalized channels for some of the biggest libraries. Migrating Official Channels into DiscordAfter the discord settled, we migrated DefinitelyTyped support from Gitter into a channel in discord and created similar channels for handling contributing to the website/playground and localization. Occasional EventsWe ran a TypeScript Q&A on a release, and in theory could do that again sometime. I'm seeing other discords (like RedwoodJS or Svelte's) doing interesting things with video tutorials, workshops and live podcasts. Maybe there's space for doing events like that in the TypeScript Community discord. Would I Do It Differently Now?I think I'd still opt for something like Discord for TypeScript today, 2+ years later. The same team who were moderating when I joined are still actively running the show, so I feel like it was a good trade for them, and there's a few regulars from the DefinitelyTyped Gitter now helping out with moderation in the Discord. |
|
Wrapping up this mini-blog. Hey folks, thanks for subscribing to my issue where I kept folks in the know on what was going on in the web/community infra for TypeScript. This issue provided a great way to write up some of the smaller wins and design decisions which would typically be kept internal because it's not big enough for the main TypeScript blog. Today is my last day on the TypeScript team and so it feels like the right time to close this issue up and remove it from the website footer - when I interviewed for this role, I gave a metaphor that TypeScript is a medieval castle and there's an incredibly large village growing around it which needs some thinking about. A few years later, I'm happy to go back to being a villager instead of a town planner now that we're in a good place. I'll still be around in the TypeScript Community Discord or Twitter (@orta) if you are interested in how things are going or have questions about how it all works. |
and others
If you are interested on updates to the TypeScript web infra, you can subscribe to this issue by clicking down and to the right. I'll comments with project updates. I try to do 3 days on web infra, and 2 days on compiler things a week, so there should be some progress every week or two.
If you're on a mobile, I'd recommend only loading this page on WIFI, the GIF usage is pretty intense.
Long term roadmaps:
A lot of progress happens on the v2 branch of the TypeScript website repo, the results of this is attached to the main website under a folder called v2: https://www.typescriptlang.org/v2/
Please don't comment in this thread (you're welcome to poke me on Twitter ( @orta )) - I'm trying it unlocked to let reactions stick around so that it doesn't feel like shouting into the void.
Subscribe
About myself 👋
Hello there, I’m Orta! I'm reasonably new to the TypeScript team, and have been focusing on docs and web tooling. You can find more fine-grained updates on twitter.com/orta (plus y'know, other stuff) - also, thanks for the idea https://github.com/octokit/rest.js/issues/620
The text was updated successfully, but these errors were encountered: