Editor support for WebComponents

[octref]

Hi, this is Pine from VS Code team. I'm working on expanding our HTML Language Server to better support WebComponents. I'd like to get some feedback from the WebComponents commitee / community on implementing WC editor support.

I also work on Vue Language Server where I'm already shipping such support for Vue-based frameworks.

Current Status

https://github.com/octref/web-components-examples

• User includes web-components.json and web-component-attributes.json in a project
• User points to these JSON files with html.experimental.custom.tags and html.experimental.custom.attributes configurations
• VS Code gets HTML completion and hover for the custom tags / attributes.

Some Background

• The TypeScript community has adopted Type Definition Files, which drives the completions, hover messages and many other language features in VS Code and other editors through javascript-typescript-langserver.
• Through Language Server Protocol, HTML Language Server in VS Code can be used in other editors too.

Use Cases

The main two use cases are:

• Someone ships my-wc-libs library. He includes web-components.json in his NPM distribution (similar to packages include d.ts files for TypeScript) and puts webComponents: "./web-components.json in his package.json. VS Code finds the JSON files to enhance its HTML support.
• Someone just wrote my-component.js locally in a project. It would be awesome if an Analyzer (non-existent yet) would figure out the customElements.define calls, generate such JSON files and feed them to the HTML Language Server to enhance its functionalities. When the user navigates to a HTML file, he should get HTML completions for the WebComponent he just defined.

Discussions

I'd like to get some input for questions below. But if you have other questions / concerns, feel welcome to chime in.

• What are some additional editor features that could be implemented for Web Components? For example, jump to definition on <my-component> in HTML goes to the JS source where the component is defined?
• Would the WebComponents committee be interested in specifying a format for such JSON files?
• Would WebComponents implement an Analyzer, similar to Polymer Analyzer for generating such JSON files on-the-fly?

[adamdbradley]

I think this is a great discussion to start. With stenciljs, the compiler is running static analysis on all of the components, then auto-generating a json file, JSX type definitions, and adding to typescript's lib.dom.d.ts global HTMLElementTagNameMap. Our json file sounds exactly what you are describing, so it'd be great to follow a standard, or at least a recommendation so it works well with external tooling.

[justinfagnani]

This is awesome to see :) I have a bunch of thoughts in this area, since we've been working on similar things for a few years.

First, is that elements have a very right API surface, not just limited to attributes, and for the best experience we need to take all of them into account:

• Tag name
• Attributes
• DOM interface: Properties, methods, superclass, implemented interfaces
• Events fired
• CSS Variables used
• CSS Shadow Parts/Themes exposed
• Slots
• Permitted content
• Permitted parents
• Default / Permitted ARIA roles

I think the easiest way to describe the DOM interface if via .d.ts files and HTMLElementTagNameMap, and JSON is probably ok for the rest, modulo some cross-file references like the types of Events. I hope that other IDEs and toolchains would be ok with using .d.ts for the interface description.

The Polymer Analyzer generates a JSON file already that describes a lot of this information for elements, inferred from the prototype chain and jsdoc annotations. The Analyzer is not very Polymer specific in most places, aside from some bundled plugins to recognize Polymer-specific declarations. It does analyze vanilla elements that extend from HTMLElement and/or are registered with plain customElements.define().

We would be very interested in contributing infrastructure to a non-Polymer-branded analyzer that VS Code and the rest of the ecosystem can easily use. We would also be very interested in reading this information from package.json and using it to drive webcomponents.org, rather than doing analysis on published npm packages.

Given that the typings, definitions, and jsdoc tags are interop points between tools, I think we should start with defining their formats. Then on our end we can work on updating the Polymer Analyzer to output this format and support new jsdoc tags rather quickly, and webcomponents.org to read the formats.

Note: We've started looking at adding better analysis support to the lit-html VS Code extension, and the biggest immediate roadblock is the lack of integration between the HTML and TypeScript Language Services. The HTML Language Service doesn't recognize that a tag name maps to a class, so it looks like we'd have to just re-parse the HTML in a separate service. cc @rictic there.

[arjunyel]

It would be cool if the element metadata was available via JSON as well :D

Now that WASM supports DOM access I imagine it would be possible for WASM languages like Rust to parse that JSON and offer type completion when working with custom elements

[PaulHMason]

I'm more in favour of self-describing web components (even if that means just attaching the json description file content as a static property to the class definition), that way browser dev tools could also be a bit smarter when working with web components (but I suppose they could work with the external files in a similar way to source maps). I also think it's worth considering how web components could be made more compatible with design surfaces - being able to register extended information, scoped registries and being able to remove registered custom elements would make things a lot easier.

[manucorporat]

@justinfagnani
I agree, we could and should be able to include more useful information, like CSS vars that apply (CSS autocompletion), the "slots" provided by a component or even the "type" of an attribute. While we all know HTML attributes might only be strings. The valid values differ:

• Any string value
• "true" | "false"
• "on" | "off"
• any union of strings (side: 'left' | 'right')
• if an attribute is required or not

I think the .json is looking good, no need to ship all this features the first day, very promising!

@PaulHMason

(even if that means just attaching the json description file content as a static property to the class definition)

I don't think we should make web components bigger to fix a DX issue, not a runtime one. It's a matter of tooling, polymer, stencil or even VScode itself could parse web components and "auto-generate" whatever the metadata is required.

@octref
Question, why do we need "two" .jsons?

[manucorporat]

btw, it's working!!

(ionic-team/stencil#1256)

[willi84]

Apropos webcomponents: maybe in the same view it would be cool to give support for inline templates (like in angular) to provide the html featureset of auto completion, highlighting etc for custom selectors or web components

[pjmolina]

Hi!
Joining the discussion.

I fully agree we need to come up with a standard way to expose types for Web Components and love to see more people demanding it. This will enable tooling support for WebComponents.
My two cents on it: A working draft for a Spec: Typed Web Components Spec
Not complete, (needs to add Slots and more) but accepting Pull Request ;-)

It is inspired by OpenAPI to provide a JSON/schema spec to be language and framework agnostic.
But at the same time, we can derive *.d.ts files or viceversa for example to leveraged typed languages.

[caridy]

Great stuff. My only immediate feedback is to not focus on customElements.define() via Analyzer, mainly because these two reasons:

• many folks will just let the registration process to be mechanical as part of the build process rather than explicitly authored, which means that you author a class with all the stuff in there but the definition of the tagName associated to that class is not something that you have to author.

• many frameworks will be able to compile to web-components, (e.g.: VUE or React), which is a process that will happen as part of the compilation step.

Having an out-of-band configuration file to describe the shape of your component, and the inheritance seems to be good enough.

[octref]

@adamdbradley Do you have a pointer to how you are using HTMLElementTagNameMap?

@manucorporat Thanks for the quick PR. Seems stencil is already generating metadata for compilers and you just added one more JSON output? Glad to see such thing baked into frameworks.

"two" .jsons?

No specific reasons. I don't know what setting to call if I combine both. And there are frameworks like Mavo which adds only global attributes but not tags.

WASM

@arjunyel Yep. Each language targeting web will embed HTML (like JSX) or become embedded by HTML (like Vue). These metadata provide foundation for building Language Service to support those cases.

self-describing web components

@PaulHMason You can see the discussion I brought up for Vue: vuejs/vue#7186. I think we should leave this discussion to each framework. As long as they generate JSON of a specific format, VS Code can use it to enhance the HTML editing capabilities. For WC, I don't know enough on this matter to give an opinion. For Vue, it depends on what experience we to give for TS.

types for Web Components

@pjmolina I don't know enough about WC to give opinions on this matter. Bringing types to Vue has been very difficult, though.

not focus on customElements.define()

@caridy That's correct. However we should have one that can handle build outputs (glad @justinfagnani is willing to help here). This is useful for people shipping WC libraries.

It's impossible for VS Code to support each framework. Our approach has been offering building blocks (like vscode-html-languageservice and let framework Language Servers build upon on them (lit-html, Vetur). The html-language-service will expose APIs for loading additional tags/attributes, and I can imagine a React analyzer generate metadata for its components, feed them to html-language-service to enhance lit-html.

[octref]

We would be very interested in contributing infrastructure to a non-Polymer-branded analyzer that VS Code and the rest of the ecosystem can easily use. We would also be very interested in reading this information from package.json and using it to drive webcomponents.org, rather than doing analysis on published npm packages.

@justinfagnani That's awesome to hear! Thanks.

I think your long list is "JSON metadata WC should generate for empowering other tools", which would be a superset of what editor needs. Let's limit the scope of the discussions here...

For VS Code's HTML we are currently only thinking about declarative contributions (like JSON files), not JS plugins like TS Server Plugins. We might do it in the future though, so you could wire a ARIA-validation-plugin into the HTML Language Server.

Given that the typings, definitions, and jsdoc tags are interop points between tools, I think we should start with defining their formats.

I agree. We should start specifying a contract between the JSON file and editor features & LSP methods. With this you can work on improving the Analyzer and us on the HTML Language Server to support this contract.

For example:

• When JSON adds a tag definition (with attributes/self-closing-or-not and other metadata), editor should:
  • Handle LSP Completion for tag/attributes
  • Handle LSP Hover for tag/attributes (show description)
  • Provide Diagnostics if a required attribute is missing, or if a non-self-closing tag is self-closed.

Notice the JSON data might be reused for VS Code's css/emmet support too. For example, CSS selector completions should include custom tags, and emmet should expand custom tags.

We've started looking at adding better analysis support to the lit-html VS Code extension, and the biggest immediate roadblock is the lack of integration between the HTML and TypeScript Language Services

Feel free to tag me or @mjbvz in the specific issues.

[arjunyel]

I think your long list is "JSON metadata WC should generate for empowering other tools", which would be a superset of what editor needs. Let's limit the scope of the discussions here...

For the HTML language server I understand it basically just needs attributes, but for all other tooling like my WASM idea we need the other data such as the DOM properties, I'd like to see Justin's list become the spec if possible. BTW I appreciate you starting this issue, I'm excited to see where it ends up :D

[manucorporat]

@octref yeah! we have already a bunch of different outputs, like markdown, json, angular bindings, react bindings...

For the record, check out our "docs-json" format that includes all information we have:
https://unpkg.com/@ionic/[email protected]/core.json

[bkelley13]

Is there a way to provide a list of options for attributes? ie, if I have my-component, with my-attribute and my-attribute has a choice of {left|right}, could this be done?

[octref]

With 1.31 VS Code has formalized the data format a bit. We also have some docs and JSON schemas for the JSON files.
Take a look at these resources:

• Release Notes on custom data support
• HTML CustomData doc
• A simple sample and the SVG data sample

@bkelley13 Yep, that can be done with valueSet. Take a look at the valueSet in the HTML CustomData doc.

[runem]

First of all, thank you for your great work on this JSON format @octref 🎉

Right now the format has a lot of momentum. Experimental support in VS Code, the Stencil compiler generates it and it’s considered for the reboot of webcomponents.org.

I would really like to get the discussion going around expanding the format, and I hope my thoughts and suggestions will help this discussion. Here I suggest some features that I think would be a great addition for a potential next version of this JSON format. I know there are some open questions in my post here, but I'm pretty much posting these suggestions to start the discussion. Go here for TLDR.

Purpose

I would like to briefly outline the purpose of this format (at risk of repeating the first comment on this issue).

This JSON format is a common way of describing custom elements. A library shipping custom elements should include a JSON file with metadata describing custom elements provided by the library. This format can be used to give editor support, generate documentation/demos and much more.

I think the format should be simple enough to be written by hand, but a library maintainer would likely be using a tool to generate them. The tool that the maintainer chooses would depend on which library they used to build the elements (e.g. the Stencil compiler already outputs this format).

I also think it should be a goal that it’s possible to describe the API of built-in HTML elements using this format.

Suggestions

Describing members

I suggest the next version of this format supports the following 4 DOM features:

• Attributes
• Properties
• Events fired
• Slots

In my experience, these 4 features of a custom element API are the most important to know for a consumer. The existing version of this JSON format already supports attributes.

It would make sense to add 3 optional fields called properties, events and slots on the tag object next to the existing attributes field:

{
...
"tags": [
   {
      "name": "my-button",
      "description": "This is my button",
      "attributes": [...]
      "properties": [...]
      "events": [...]
      "slots": [...]
   }
],
...
}

Just like the existing attributes, I think the content of all 3 new fields should at least have a required name and an optional description.

The new properties objects should also have values and valueSet like the existing attributes.

Based on the aforementioned, the JSON would look like this:

{
...
"tags": [
    {
      "name": "my-button",
      "description": "This is my button",
      "attributes": [{ "name": "color", "values": [{ "name": "red" }, { "name": "green" }] }],
      "properties": [{ "name": "color", "values": [{ "name": "red" }, { "name": "green" }] }],
      "events": [{ "name": "colorchange", "description": "This event will fire whenever 'color' changes" }],
      "slots": [
        { "name": "", "description": "Default slot. This will become the main content of this element." },
        { "name": "header", "description": "Content that will go in the header." }
      ]
    }
]
...
}

Note that an unnamed slot has an empty string as the name because name is required for all members.

Relation between property & attribute

In the example above, two related members called color are repeated as both an attribute and a property to indicate that both can be set on this element to control the color. There is however no explicit connection between them in the JSON right now.

In order to describe the relation between a property and an attribute, I think we need to look into the following:

1. A way to describe which attribute a property is related to. e.g. a property called "myText" could describe the same data as an attribute called "my-text".
2. A way to describe how setting one will affect the other (reflection).

To solve (1) I suggest adding attribute: <string> field that can be used when describing properties. This field would indicate that property and attribute describe the same data. This field only indicates the link. In addition, I think it should be optional to specify the corresponding attribute in the attributes array in order to stay DRY.

To solve (2) I suggest adding reflect: <"both"|"to-attribute"|"to-property"> field that can be used when describing properties. Setting reflect would indicate that updating the value of either the attribute or property could also affect the value of the other. I think that when using reflect it should be required that attribute is also present.

{
...
"tags": [
   {
      "name": "my-button",
      "description": "This is my button",
      "properties": [{
         "name": "myColor",
         "description": "This is my color",
         "attribute": "my-color",
         "reflect": "to-property"
      }]
   }
],
...
}

Describing types

The existing version of this format only supports string enumeration types (by using values arrays). This makes sense as all attributes in the DOM are strings. In addition (correct me if I’m wrong @octref) it seems like boolean attributes are using the valueSet: 'v' even though "v" doesn’t exist as a value set?

I think this format will need to have some way of specifying more complex types than string enumeration types. This is applicable for both attributes and properties.

Even though all attributes are strings as far as the DOM is concerned, I think it would be very useful to tell what type of value the attribute is coerced/converted to. Imagine reading the documentation about an element or using an IDE-plugin to autocomplete attributes: Here it would be great to know that the value given to e.g. max is coerced to a number or that the disabled attribute is a boolean attribute.

Let's look at some examples from built-in elements: ‘max’ on <input> is coerced to number and required on <input> is a boolean attribute, so coerced to a boolean. Another example is that the download on <a> behaves both as a boolean attribute and also accepts any string value (which becomes a prefilled file name). In this case, I think it would be powerful to tell that the type of this attribute is a union of boolean and string.

With all this in mind, I suggest that we explore adding an optional type field to the JSON format. This field can be used to describe the types of attributes, properties and events. I think it would make sense that the format of the type field is a jsdoc type expression, because then we avoid inventing a new type format. The type of the download attribute would then be: "(boolean|string)".

Here is an example of how it could look:

{
...
"tags": [
   {
      "name": "my-button",
      "description": "This is my button",
      "properties": [{
         "name": "prop",
         "description": "This is my property",
         "type": "(string|boolean)"
      }]
   }
],
...
}

In the following sections, I will explore what using jsdoc gives us and the challenges that follow. It could also be interesting to investigate if it makes sense to use Typescript definition files instead, but I fear that making the JSON format dependent on Typescript for types would not be the right way to go. Therefore I chose to go with jsdoc type expressions, but there might be something even better. I'm very interested in hearing what you think.

I'm aware that this opens a lot of questions that will need to be considered. Most notably these two questions which I will try to address in the next sections:

1. If this is a type expression, how would we define interfaces defined with typedef or Typescript interfaces?
2. This would make values redundant since type now covers most use cases offered by it. Would it be valuable to deprecate values?

values and type

As mentioned before values already exist. This opens the question if it would make sense to deprecate this property. I'm in doubt.

A string enumeration could with type be expressed like "type": "('red'|'green'), but with values it could be expressed like "values": [{"name": "red", "description": "Makes the element red"}, {"name": "green"}]. The big difference is that it’s possible for each value in values to have a description thus making it possible to document what each value of the string enumeration does. Therefore type doesn't entirely replace the functionality of values.

valueSet

We could also allow adding type to valueSet like this so it becomes possible to have shared types across the JSON file:

{
...
"valueSets": [
   {
      "name": "colors",
      "type": "('red'|'green'|'blue')"
   }
],
"tags": [
   {
      "name": "my-button",
      "description": "This is my button",
      "attributes": [{
         "name": "color",
         "valueSet": "colors"
      }, {
         "name": "color-hover",
         "valueSet": "colors"
      }]
   }
],
...
}

Complex types

Supporting complex types comes with some challenges. This is the point where I'm the most in doubt, so I look very much forward to hear what you think.

Let me give an example of a challenge: If we describe a property called color of type being a jsdoc type definition (or Typescript interface) called Color (with r,g,b properties), it would result in the color property (in the JSON file) having the type "Color". The details of the type definition would be gone and we would now only know the name of the type, not the structure. If necessary, a tool parsing this JSON file, reading the type Color, would have to find the type with that name somewhere else (e.g. a d.ts file) or just print out that the type is Color. The question is how.

1. A possible solution could be a tool generating these files inlines the jsdoc type definitions like this: "{r: number, g: number, b: number}". The downside when inlining type definitions is that we lose the name of the type.
2. Another solution would be to support jsdoc type imports like import("./src/Foo.js').Bar. The path would be relative to the JSON file. A tool analyzing the JSON file could now either look for a JSDoc type definition named "Bar" in "Foo.js" or a ts/d.ts file "Foo.d.ts" exporting a type named "Bar".

Types and events

The type of events would describe the type of the dispatched event. Here are some examples:

• Event: This is the most general event type.
• MouseEvent: The built-in mouse event.
• CustomEvent<number>: This would be used if you dispatch a custom event. The generic type is the type of detail.
• MyChangeEvent: This could be used if you extend Event and dispatch your own event called MyChangeEvent. It leads us into the same territory as described above with Complex types. Where do you define what kind of event MyChangeEvent is? Again, this could be solved by using a jsdoc type import like import("./src/Foo.js').MyChangeEvent.

Default property values

Properties usually are holding the "local" state of a custom element, therefore, it is highly interesting for users to know these default values/states.

Attributes do NOT have a default as users are providing the html. However, when using the combination of a default value and reflect, then a custom element will get set a certain attribute if not already defined by the user.

We could add default: <string> when describing properties like this:

{
...
"tags": [
   {
      "name": "my-user",
      "properties": [{
         "name": "color",
         "default": "red"
      }]
   }
],
...
}

Deprecated

It would be useful to add deprecated: <boolean | string> to members and tags. This makes it possible to provide a clear path forward when APIs are going to change in an upcoming breaking release.

A tag gets deprecated

{
...
"tags": [
   {
      "name": "my-button",
      "deprecated": "Please use <other-button> instead.",
      ...
   }
],
...
}

A property or attribute gets deprecated

{
...
"tags": [
   {
      "name": "my-user",
      "properties": [{
         "name": "name",
         "deprecated": "Use .fullName instead",
      }, {
         "name": "fullName",
      }]
   }
],
...
}

Examples

In JSDoc a well-used feature is @example allowing to provide copyable code snippets for your users. We can extract that to be easily usable by documentation tools. Can be used for members and for tags.

The field to describe examples could be example: <string[]>. It's an array as multiple examples should be possible (like in JSDoc). If there is only one example then array will have a length of 1.

{
...
"tags": [
   {
      "name": "my-user",
      "properties": [{
         "name": "fullName",
         "example": ["el.fullName = ‘foo bar’;\n// this will make it available via\n console.log(el.firstName);\n console.log(el.lastName);", "another example"]
      }]
   }
],
...
}

Methods

It would be very useful if it was possible to document methods of a custom element using this JSON format. When using and extending custom elements it's rather important to know which methods are available and therefore having them available for documentation would be a big plus.

This can be done by adding methods to tags. The value of methods would be an array of method objects. Each method object would describe a method with a required name, optional description and an optional params array. Each param would consist of an optional jsdoc type expression type and an optional description.

This is an example of how it could look like:

{
...
"tags": [
   {
     "name": "my-dialog",
     "methods": [{
        "name": "open",
        "description": "Open the dialog",
        "params": [
          { "type": "String", "description": "Override title" }, 
          { "type": "HTMLElement", "description": "Element to focus after close" }
        ],
      }]
   }
],
...
}

Summarizing properties

If these suggestions are accepted, properties will become the entry with the most info. e.g. it will have additionally “default”, “type”, “attribute” and “reflect”:

{
...
"tags": [
   {
      "name": "my-user",
      "properties": [{
         "name": "colorName",
         "type": "String",
         "default": "red",
         "values": [{ "name": "red" }, { "name": "green" }] }],
         "attribute": "color-name",
         "reflect": "to-attribute"
      }]
   }
],
...
}

Path

I suggest adding path: <string> to tags. This field would describe where to find the element. The path could be a link/url/file, and the relative path would be relative from the JSON file.

This information would be useful to e.g. IDE-plugins and allows to find source files and Typescript definition files.

{
...
"tags": [
   {
      "name": "my-button",
      "description": "This is my button",
      "path": "./src/my-button.js"
      ...
   }
],
...
}

Globals

In the existing vscode JSON format, global attributes can be described using globalAttributtes on the root object. This field would primarily be used if one were to describe existing, built-in elements, but I could also see a library shipping custom elements describing global attributes for styling purposes.

In addition to global attributes (like the ‘title’ attribute), there also exists global events (like the ‘click’ event).

Therefore I suggest that the existing globalAttributes is removed and replaced with global like this:

{
...
  "global": {
      "attributes": [...]
      "properties": [...]
      "events": [...]
      "slots": [...]
  }
...
}

Note that this would be a breaking change.

I chose to include slots and properties in global for completeness sake. Right now I’m not sure what it gives of value, but I like that all of the 4 features I mentioned before can also be described in global to give more flexibility.

Sharing the JSON file

I have heard two different suggestions of what to call the JSON file: web-components.json and custom-elements.json. Personally I like the latter best because it’s broader and because the only requirement is that you describe custom elements.

I also think there could be value in being able to point to this file using a "customElements" field in package.json. The field could accept both a single path and an array of file paths: "customElements": "./custom-elements.json" or "customElements": ["./src/my-button/my-button.json", "./src/my-input/my-input.json"]. Describing custom elements using multiple files could make it easier if one were to maintain them by hand.

However, I'm not sure if adding customElements to package.json should be a requirement. Tools could just as well look for custom-elements.json in the root of a package.

Conclusion

Overall we would like to encourage people to publish their (npm) packages including a custom-elements.json.

The file should look something like this:

{
...
"tags": [
    {
      "name": "my-button",
      "description": "This is my button",
      "attributes": [{ "name": "disabled", "type": "boolean" }],
      "properties": [{ 
        "name": "colorName",
        "default": "red",
        "type": "('red'|'green'|'blue')",
        "attribute”: "color-name",
        "reflect": "both",
        "deprecated": "Use .fanzyColor instead",
        "example": ["a code example in html or js"],
      }],
      "events": [{
        "name": "colorchange", 
        "description": "This event will fire whenever 'color' changes" 
      }],
      "slots": [
        { 
          "name": "", 
          "description": "Default slot. This will become the main content of this element." 
        },
        { 
          "name": "header", 
          "description": "Content that will go in the header." 
        }
      ],
      
      /* following is maybe worth adding - or wait for next iteration */
      "methods": [{
        "name": "open",
        "description": "Open the dialog",
        "params": [
          { "type": "String", "description": "Override title" }, 
          { "type": "HTMLElement", "description": "Element to focus after close" }
        ],
      ]}
    }
]
...
}

I tried to suggest as few additions/changes to the format which I think will result in the greatest improvement. The purpose of this post is to start the discussion around the format with some concrete suggestions and considerations. I'm sure that there are many questions still left to consider and perhaps not all suggested additions are needed.

I added experimental support for some of what I suggest in this comment to web-component-analyzer. I will add more features to the experimental support during the next couple of days. This way you can get an idea of how this format would look based on your own custom elements:

npx web-component-analyzer analyze my-element.js --format json --outFile custom-elements.json

Note: you can also use --format vscode to generate the existing JSON format.

Thanks for reading this comment. I look very much forward to hear what you think about these suggestions.

[justinfagnani]

@runem awesome write up :)

I think that we should open a repo somewhere, possibly in the /webcomponents/ GitHub org, to specify the custom-elements.json format. Then we can have a page the the format description, multiple issues, PRs, etc.

Also, as a really high-level point about the format: The set of information we need for web components is a superset of what's needed for JavaScript/TypeScript in general. This is especially clear when you consider class methods and properties and their types. Web components are just classes with extra structured API surfaces.

I'm not aware of any well-specified format for JS docs, but the output format of the Microsoft API Extractor seems like a good start to me: https://www.npmjs.com/package/@microsoft/api-extractor It would be ideal to me to have both a specified JS/TS analysis format, and a web components extension to it.

[sghoweri]

@runem WOW - this is awesome!!

I’m still digging into all this but at a high level, this looks like it’ll solve a ton of the growing pains I know me and my team are working through as our design system continues to ship almost everything as a web component...

Up till now we’ve primarily used JSON schemas to document / manage APIs (especially when dealing with backend Twig integration). Unfortunately, the JSON Schema spec just can’t cover the deeper documentation needs of Web Components: deprecating props, events, methods, slots, CSS custom properties, etc — so I’m SUPER excited to see a more standardized (+ IDE-supported
) way to get all the same benefits and much much more.

Can’t wait to dive more into this (plus the web component analyzer tool)! 😊

[octref]

Hey @runem, thanks for the writeup. To your suggestions, I'll have a more detailed response later — preparing for a conference and boarding plane soon. But here are some high-level notes:

• I suggest starting smaller and only adding fields that address real need, serve clear purpose & have no controversy/alternative design. It's always easier to add a field later, but it can hurt to change/remove a field when others are already depending on it.
• I'd stay close to spec. For example adding one more field to describe a CSS property based on CSS Value definition syntax. Inventing a custom syntax and type system isn't feasible.
• The design of LSP avoided being too strict to define every UX interaction that can happen in the LSP methods, but not too loose to give guidance for editor UI implementations. It also provided a reference implementation. I think that's a good model to learn from. Before expanding the format too much, I hope the tooling is ready. Your analyzer would help a lot. But what about a doc generator that can generate doc from the data format?

Some other notes:

Experimental support in VS Code

We marked it stable last month. Here's the repo: https://github.com/microsoft/vscode-custom-data

The format is versioned and we plan to roughly follow SemVer. Now the development of this custom data format is mostly driven by editor language features such as completion, hover information etc. There's no consumer generating docs from such data so we haven't considered fields that would be useful for that purpose.

describe the API of built-in HTML elements using this format.

That's what we are doing already. See https://github.com/microsoft/vscode-custom-data.
The /web-data package is published to https://www.npmjs.com/package/vscode-web-custom-data. The data is aggregated from MDN, HTML spec, WAI-ARIA spec, etc.

[daKmoR]

Is the example web-components.json the latest version? it uses "label" instead of "name"? could be that I missed that change 🙈

imho the minimum to make it useful for more than "just" vscode depends on what it is used for

Use Case Docs:

What web component users are interested

• custom-element name
• attribute names
• property names
• events names
• slot names
• css variable names
• deprecations, defaults, examples ... (e.g. common info usually part of jsDoc)

and adding a short description for all those "names"/"labels"

basically outline what I can expect from this custom element / web component.

=> web-component-analyzer can already output markdown documentation. Right now it's using the "internal" data for it... but it would be nice if custom-elements.json as input would be all thats needed.

Use Case Web Component Catalog:

Allow searching ala npm search but for web components only.
e.g. npm package that has a custom-elements.json will be treated as web component and get's index

info needed is the same as for Docs (e.g. index all the names/labels/descriptions and let the user search through it all)
=> I am currently working on that 👍

Use Case Playground:

An example would be storybook with can render a web component and give knobs to modify all the states of it.

It needs

• custom-element name
• attribute names + Knob type
• property names + Knob type

A description is not needed in that case; A Knob Type can in many cases be derived from the actual type. Still, there will need to be a method of overriding it.

Suggestion

add a property jsDoc which "just" contains the raw jsDoc from the code if available.
Then examples, default values, Knob Type, type, ... can all be custom parsed if needed. (but still, only custom-elements.json is needed as a source of input - no source code parsing is required)

"properties": [
  {
    "label": "text",
    "description": "The text inside the component",
    "jsDoc": "/**\n Some longer description\n * @type {string} * @example ... \n * @storybook-knob ColorPicker\n */"
   }
]

That way the json format stays simple and tools use cases can still get the information they need. If at some point everyone uses the same jsDoc we can make it a separate property in the json.

Possible ways forward

adding minimal:

• properties (with label, description, jsDoc)

adding for docs (e.g. above +):

• events (with label + description)
• slots (with name + description) [use name as slots have names not labels?]
• css variables?

adding full (e.g. above +):

• add types
• link properties to attributes (with attribute and reflect)
• extract examples, type, default, for properties

Example for docs:

{
  "tags": [
    {
      "label": "open-shadow",
      "description": "Open Shadow DOM Component",
      "properties": [
        {
          "label": "text",
          "description": "The text inside the component",
          "jsDoc": "/**\n Some longer description\n * @type {string} * @example ... \n * @storybook-knob ColorPicker\n */"
        }
      ],
      "events": [
        {
          "label": "colorchange",
          "description": "This event will fire whenever 'color' changes"
        }
      ],
      "slots": [
        {
          "name": "",
          "description": "Main content"
        },
        {
          "name": "left",
          "description": "Left content"
        }
      ]
    }
  ]
}

[mzeiher]

Wow this looks awesome, we use a similar file to describe all our components where I work, If this takes traction, we can switch to this standard. One additional information would be nice, we use a lot of inheritance for our custom elements (internally we use something similar to lit-element) and also make use of extending built-in elements for progressive enhancement.
We came up with a field "superClass" which specifies the identifier of the super class and where it's from
example:

... for relative paths
"superClass": {
            "identifier": "DropDown",
            "from": "../drop-down/DropDown.js"
},
... for components in other packages
"superClass": {
            "identifier": "DropDown",
            "from": "my-base-package/lib/DropDown.js"
},
... for built-in elements
"superClass": {
            "identifier": "HTMLBUttonElement"
},

for our tooling we also add the identifier of the exported class which represents the custom element and the path of the js module that exports the class (path relative to our components.js)

[daKmoR]

So we are working on a catalog to index web components and the idea is to add every package that has a custom-elements.json. So could we agree on the filename?

PS: web-components.json sounds fine as well => as long as we agree on a name and keep it 🤗

[AndreasGalster]

Properties make sense when you only want to write to property, e. g. lit-element has syntax to either write to attribute or to a property. It's quite common if you don't need the property to reflect an attribute on the element.

For events, I suppose autocompletion for on-event="doThs() syntax makes sense. Libraries like lit-element also have custom on-event-name syntax. It would be great if libraries could pick up the events so they can be autocompleted.

My personal take on this issue is that if there can be one source of truth for as much documentation for a component, that would be great. I understand your goal is to make editor support via autocompletion easier, and I agree that's a great endeavor. Though some way to auto-generate docs outside a code editor from this would be amazing.

I can see that methods don't make sense for the code editor auto-completion but they'd be very beneficial for general docs.

[justinfagnani]

@daKmoR

as another consideration there is web-types...

https://github.com/JetBrains/web-types

so how are we going to move forward?

I hadn't seen this one. I think it's a little unfortunate to have repos spawn on separate IDE owners orgs to try to create standards that span many tools beyond IDEs.

I think at this point we need a central repo to work up a spec. I can open one in https://github.com/webcomponents and then we can have individual issues for each topic and organize the conversations much better.

[justinfagnani]

Ok, I created a new repo at https://github.com/webcomponents/custom-elements-json

I made a few seed issues around goals, requirements and such. It would be great if we could gather a good discussion there.

[bahrus]

Looking at the example syntax for web-types: https://github.com/JetBrains/web-types/blob/master/examples/vue.web-types.json,

It looks to me that the two schemas aren't that far apart. It looks like the "html" section is quite close to the VSCode format, and even with the https://github.com/runem/web-component-analyzer format, only far more exhaustive. I don't see anything in that section that contradicts the VSCode format, only more stuff. I might be missing something.

So kind of good news, maybe?

[daKmoR]

Storybook API Table

Storybook now supports an api table in docs mode for your components which is also powered by a custom-elements.json which you only need to setup once.

and then you can use it via <Props of="demo-wc-card" /> in their mdx docs.

See the following demo.

Storybook API Playground

You can also auto-generate an "api" playground - again based on metadata within custom-element.json. See the demo.

Thoughts

The used custom-elements.json in that example is not too different.

I would consider it a subset of what is currently in vscode.

For documentation and api playground there is more/different information needed then when doing autocomplete in vscode. However, as long as the data is not contradicting each other we should be fine right?

The types used in the example are more "typeHints" e.g. for documentation so that users can read what the name of the type is that is expected. Or what kind of value the css variable is used for. It's not really a "true" type check or so...

Web Component API viewer

There is also a very nice project api-viewer-element. Which generates full API documentation and API Playground purely based on a custom-elements.json.

Again it adds "typeHints" and additionally jsDoc

See the live example

Resume

A lot of stuff is happening and it seems to be really useful to add more info (besides what is truely needed in vscode). As long as it is considered a subset it should be fine for everyone right 🤗

Sooo enough for now let's head over to https://github.com/webcomponents/custom-elements-json and spec that thing before people go too crazy 😬

[octref]

I would consider it a subset of what is currently in vscode.

@daKmoR I'd say it's a superset. Think of it this way:

• slots, events etc are specific to your WC. You compile them together into HTML API doc, and browser render it.
• Similarly, you compile description, slots, events into a Markdown and give that to VS Code as the description field, which renders that in its hover and auto-completion widgets.
• I think keep the bridge just the Markdown formatted description is good. There's a 1-1 relationship between what data you feed VS Code & what you see on UI. Saying "now VS Code's custom data format would include slots, events, cssProperties and dictate how they should be rendered" limits your freedom to organize the message in the auto-completion widget, and these don't apply to scenarios outside WC.

[daKmoR]

@octref thx for clarifying on the superset - I fully agree 👍

also, I fully agree on that does not make sense to add slots, events, cssProperties to vscode-custom-data.
Let's only add them to custom-elements.json - as there we need it for API docs and such 👍

Looking back at the thread I have the feeling there have been some missundestandings 🙈 as far as I know it has never been the goal to bring all web component meta data into vscode-custom-data 🤔 more to bring them into a common file which would not contradict the vscode-custom-data...

I'm sorry if there was confusing but I have a feeling it's now on the right track 🤗 thx for bearing with me/us 🤗

[castastrophe]

There are also "alternatives" - for example, @castastrophe let me know that they are using a JSON schema to document their web components. Here is an example https://github.com/patternfly/patternfly-elements/blob/master/elements/pfe-band/src/pfe-band.json

@daKmoR Thanks for the mention! So we are actually walking away from our schemas at the moment. We're implementing a built-in property management system in our base class and as such, the JS of the custom element is now the source of truth for the attributes, slots, and metadata. Going forward, you'll be able to query: PfeFoo.properties, PfeFoo.slots, PfeFoo.meta, for information about the component. This was a critical update for us as it allowed our base class to manage prefixing, observer, and cascading. cc @mwcz @kylebuch8

[piotrtomiak]

@daKmoR @justinfagnani Looks like I am a little bit late to the party :/ I wish someone had pinged me on this topic earlier. I've finally moved to the next phase of improving web-types to be a de facto standard for any web-framework, not just Vue.js - we had to evaluate it's usefulness and gather feedback before moving further. I can see that a lot has happend in Web Components community in the meanwhile. What is the currently accepted format for describing WebComponents library?

@justinfagnani Creating yet another standard makes it really hard for each IDE and documentation tool to support every framework out there. Can we work on merging your schema with web-types schema? Our goal as JetBrains is not to force any standard on the community, but to drive community to accept a single format for exchanging information about JavaScript library contents, which we believe will benefit everyone. If you don't feel that hosting an open source repo under JetBrains org is good for community we are open for transferring the repo under some community organization. As I see now, custom-elements.json being shipped with some Web Component libraries pretty closely resemble web-types JSON. We have resources to support yet another JSON format, but what about other lightweight text-editors?

[lpj145]

@piotrtomiak i already see web-types from jetbrains and i like so much what yours done, in the same way custom-elements.json as pretty good too, but we had to admit maybe we enter in a trouble (or we're already), because any ide supports and implement your own way, what guide us back in history to wars of browsers... i think is so best if we have only one way to supports ide, so i guess we can make project on rust or any other language to implement global library component analyzer for majority of libraries like: angular, vue, react, svelte, lit-element and so on...

[piotrtomiak]

@lpj145 Web-Types or custom-elements manifest is just some metadata file to describe what's in the library. We are continuing our work on Web-Types, since it solves more than just one case - it works for Angular, Vue, Web Components, React, Svelte, Font-awesome Icons, Htmx etc.. A main feature of Web-Types are patterns, which allow to describe a lot of framework features without coding. We plan to open-source the code, which allows to evaluate patterns, once it matures enough, and could be reused by other IDEs.

[MSFTKonnor]

I've been reading through and I'm curious if there's been any consideration for dynamic tag names. (It's a long thread, maybe I missed it) At work, we use prefixes to allow multiple versions of our WC library to be run at the same time.

here's an example index.d.ts I've been using with Preact that gives me full autocomplete for regular HTML tags and was curious if it'd be possible with the JSON files proposed.

https://gist.github.com/MSFTKonnor/a28d93cd88240a24cba884d5a77be9e8

or am I looking at needing to run a custom language server to generate the file htmlData JSON files on the fly that can account for different namespaces?

[break-stuff]

I know this is an old thread, but has there been any movement on this ideation?

[max529]

I've created a framework based on vscode's server protocol language so that it can be integrated everywhere. A vscode extension is already available. It allows you to generate web components based on Typescript, HTML and SCSS languages.

The aim is to be able to easily make custom attributes and tags available during autocompletion.

If you're interested I'll be happy to send you the link (or if you search my github profile you'll find it) but I want to avoid self-promotion here because that's not the aim of this community.

[break-stuff]

@max529, that sounds really interesting!

I have created some tools to generate the appropriate data files for VS Code and JetBrains IDEs (web-types) based on the Custom Elements Manifest. The thing I wish existed is the ability for VS Code to auto-detect the config files and apply them to the editor automatically. JetBrains IDEs already do this with web-types and it's an excellent way for developers to get up and running quickly with custom element libraries.

[break-stuff]

I also wish VS Code had better custom element/emmet integration. I don't believe emmet works for custom elements.

[max529]

I'm having trouble implementing emmet in my extension too.

The html.customData is a problem when working live. You have to reload the window each time to have the autocompletion up to date, which isn't cool when working on large projects.

The problem with simply installing a custom element library is that auto-completion has to work in every language that can write html (html, jsx, svelte, etc.), which is a lot of work.

And people who already use a framework don't see the point of using a native web component library outside of their framework.

If you find my idea interesting or if you want to share more about the possible integration of the Custom Element Manifest in VSCode, you can write to me privately (my email is on my github profile) to discuss it.

[trusktr]

@octref Hello, I'm late to this conversation, but I'm super interested in support for flavors of html template tags. Have you (or anyone else here) worked on making this possible for different framework's html template tags?

For example, I use (and contribute to) the Solid.js html template tag in my custom elements, and would like type checking in the templates. Here's sample template in one of my custom elements:

https://github.com/lume/lume/blob/b0c2cdcf8498fae1280c8121160ac6ff328c1588/src/cameras/CameraRig.ts#L207-L238

I have type checking support when using JSX, by wiring type defs into React/Solid/etc JSX interfaces. But I currently have no type checking when using html template tags. Solid's html works a little diffferently from Lit's, so it would need slightly different implementation.