Editor support for WebComponents

[octref]

Hi, this is Pine from VS Code team. I'm working on expanding our HTML Language Server to better support WebComponents. I'd like to get some feedback from the WebComponents commitee / community on implementing WC editor support.

I also work on Vue Language Server where I'm already shipping such support for Vue-based frameworks.

Current Status

https://github.com/octref/web-components-examples

• User includes web-components.json and web-component-attributes.json in a project
• User points to these JSON files with html.experimental.custom.tags and html.experimental.custom.attributes configurations
• VS Code gets HTML completion and hover for the custom tags / attributes.

Some Background

• The TypeScript community has adopted Type Definition Files, which drives the completions, hover messages and many other language features in VS Code and other editors through javascript-typescript-langserver.
• Through Language Server Protocol, HTML Language Server in VS Code can be used in other editors too.

Use Cases

The main two use cases are:

• Someone ships my-wc-libs library. He includes web-components.json in his NPM distribution (similar to packages include d.ts files for TypeScript) and puts webComponents: "./web-components.json in his package.json. VS Code finds the JSON files to enhance its HTML support.
• Someone just wrote my-component.js locally in a project. It would be awesome if an Analyzer (non-existent yet) would figure out the customElements.define calls, generate such JSON files and feed them to the HTML Language Server to enhance its functionalities. When the user navigates to a HTML file, he should get HTML completions for the WebComponent he just defined.

Discussions

I'd like to get some input for questions below. But if you have other questions / concerns, feel welcome to chime in.

• What are some additional editor features that could be implemented for Web Components? For example, jump to definition on <my-component> in HTML goes to the JS source where the component is defined?
• Would the WebComponents committee be interested in specifying a format for such JSON files?
• Would WebComponents implement an Analyzer, similar to Polymer Analyzer for generating such JSON files on-the-fly?

[adamdbradley]

I think this is a great discussion to start. With stenciljs, the compiler is running static analysis on all of the components, then auto-generating a json file, JSX type definitions, and adding to typescript's lib.dom.d.ts global HTMLElementTagNameMap. Our json file sounds exactly what you are describing, so it'd be great to follow a standard, or at least a recommendation so it works well with external tooling.

[justinfagnani]

This is awesome to see :) I have a bunch of thoughts in this area, since we've been working on similar things for a few years.

First, is that elements have a very right API surface, not just limited to attributes, and for the best experience we need to take all of them into account:

• Tag name
• Attributes
• DOM interface: Properties, methods, superclass, implemented interfaces
• Events fired
• CSS Variables used
• CSS Shadow Parts/Themes exposed
• Slots
• Permitted content
• Permitted parents
• Default / Permitted ARIA roles

I think the easiest way to describe the DOM interface if via .d.ts files and HTMLElementTagNameMap, and JSON is probably ok for the rest, modulo some cross-file references like the types of Events. I hope that other IDEs and toolchains would be ok with using .d.ts for the interface description.

The Polymer Analyzer generates a JSON file already that describes a lot of this information for elements, inferred from the prototype chain and jsdoc annotations. The Analyzer is not very Polymer specific in most places, aside from some bundled plugins to recognize Polymer-specific declarations. It does analyze vanilla elements that extend from HTMLElement and/or are registered with plain customElements.define().

We would be very interested in contributing infrastructure to a non-Polymer-branded analyzer that VS Code and the rest of the ecosystem can easily use. We would also be very interested in reading this information from package.json and using it to drive webcomponents.org, rather than doing analysis on published npm packages.

Given that the typings, definitions, and jsdoc tags are interop points between tools, I think we should start with defining their formats. Then on our end we can work on updating the Polymer Analyzer to output this format and support new jsdoc tags rather quickly, and webcomponents.org to read the formats.

Note: We've started looking at adding better analysis support to the lit-html VS Code extension, and the biggest immediate roadblock is the lack of integration between the HTML and TypeScript Language Services. The HTML Language Service doesn't recognize that a tag name maps to a class, so it looks like we'd have to just re-parse the HTML in a separate service. cc @rictic there.

[arjunyel]

It would be cool if the element metadata was available via JSON as well :D

Now that WASM supports DOM access I imagine it would be possible for WASM languages like Rust to parse that JSON and offer type completion when working with custom elements

[PaulHMason]

I'm more in favour of self-describing web components (even if that means just attaching the json description file content as a static property to the class definition), that way browser dev tools could also be a bit smarter when working with web components (but I suppose they could work with the external files in a similar way to source maps). I also think it's worth considering how web components could be made more compatible with design surfaces - being able to register extended information, scoped registries and being able to remove registered custom elements would make things a lot easier.

[manucorporat]

@justinfagnani
I agree, we could and should be able to include more useful information, like CSS vars that apply (CSS autocompletion), the "slots" provided by a component or even the "type" of an attribute. While we all know HTML attributes might only be strings. The valid values differ:

• Any string value
• "true" | "false"
• "on" | "off"
• any union of strings (side: 'left' | 'right')
• if an attribute is required or not

I think the .json is looking good, no need to ship all this features the first day, very promising!

@PaulHMason

(even if that means just attaching the json description file content as a static property to the class definition)

I don't think we should make web components bigger to fix a DX issue, not a runtime one. It's a matter of tooling, polymer, stencil or even VScode itself could parse web components and "auto-generate" whatever the metadata is required.

@octref
Question, why do we need "two" .jsons?

[manucorporat]

btw, it's working!!

(stenciljs/core#1256)

[willi84]

Apropos webcomponents: maybe in the same view it would be cool to give support for inline templates (like in angular) to provide the html featureset of auto completion, highlighting etc for custom selectors or web components

[pjmolina]

Hi!
Joining the discussion.

I fully agree we need to come up with a standard way to expose types for Web Components and love to see more people demanding it. This will enable tooling support for WebComponents.
My two cents on it: A working draft for a Spec: Typed Web Components Spec
Not complete, (needs to add Slots and more) but accepting Pull Request ;-)

It is inspired by OpenAPI to provide a JSON/schema spec to be language and framework agnostic.
But at the same time, we can derive *.d.ts files or viceversa for example to leveraged typed languages.

[caridy]

Great stuff. My only immediate feedback is to not focus on customElements.define() via Analyzer, mainly because these two reasons:

• many folks will just let the registration process to be mechanical as part of the build process rather than explicitly authored, which means that you author a class with all the stuff in there but the definition of the tagName associated to that class is not something that you have to author.

• many frameworks will be able to compile to web-components, (e.g.: VUE or React), which is a process that will happen as part of the compilation step.

Having an out-of-band configuration file to describe the shape of your component, and the inheritance seems to be good enough.

[octref]

@adamdbradley Do you have a pointer to how you are using HTMLElementTagNameMap?

@manucorporat Thanks for the quick PR. Seems stencil is already generating metadata for compilers and you just added one more JSON output? Glad to see such thing baked into frameworks.

"two" .jsons?

No specific reasons. I don't know what setting to call if I combine both. And there are frameworks like Mavo which adds only global attributes but not tags.

WASM

@arjunyel Yep. Each language targeting web will embed HTML (like JSX) or become embedded by HTML (like Vue). These metadata provide foundation for building Language Service to support those cases.

self-describing web components

@PaulHMason You can see the discussion I brought up for Vue: vuejs/vue#7186. I think we should leave this discussion to each framework. As long as they generate JSON of a specific format, VS Code can use it to enhance the HTML editing capabilities. For WC, I don't know enough on this matter to give an opinion. For Vue, it depends on what experience we to give for TS.

types for Web Components

@pjmolina I don't know enough about WC to give opinions on this matter. Bringing types to Vue has been very difficult, though.

not focus on customElements.define()

@caridy That's correct. However we should have one that can handle build outputs (glad @justinfagnani is willing to help here). This is useful for people shipping WC libraries.

It's impossible for VS Code to support each framework. Our approach has been offering building blocks (like vscode-html-languageservice and let framework Language Servers build upon on them (lit-html, Vetur). The html-language-service will expose APIs for loading additional tags/attributes, and I can imagine a React analyzer generate metadata for its components, feed them to html-language-service to enhance lit-html.

[octref]

We would be very interested in contributing infrastructure to a non-Polymer-branded analyzer that VS Code and the rest of the ecosystem can easily use. We would also be very interested in reading this information from package.json and using it to drive webcomponents.org, rather than doing analysis on published npm packages.

@justinfagnani That's awesome to hear! Thanks.

I think your long list is "JSON metadata WC should generate for empowering other tools", which would be a superset of what editor needs. Let's limit the scope of the discussions here...

For VS Code's HTML we are currently only thinking about declarative contributions (like JSON files), not JS plugins like TS Server Plugins. We might do it in the future though, so you could wire a ARIA-validation-plugin into the HTML Language Server.

Given that the typings, definitions, and jsdoc tags are interop points between tools, I think we should start with defining their formats.

I agree. We should start specifying a contract between the JSON file and editor features & LSP methods. With this you can work on improving the Analyzer and us on the HTML Language Server to support this contract.

For example:

• When JSON adds a tag definition (with attributes/self-closing-or-not and other metadata), editor should:
  • Handle LSP Completion for tag/attributes
  • Handle LSP Hover for tag/attributes (show description)
  • Provide Diagnostics if a required attribute is missing, or if a non-self-closing tag is self-closed.

Notice the JSON data might be reused for VS Code's css/emmet support too. For example, CSS selector completions should include custom tags, and emmet should expand custom tags.

We've started looking at adding better analysis support to the lit-html VS Code extension, and the biggest immediate roadblock is the lack of integration between the HTML and TypeScript Language Services

Feel free to tag me or @mjbvz in the specific issues.

[arjunyel]

I think your long list is "JSON metadata WC should generate for empowering other tools", which would be a superset of what editor needs. Let's limit the scope of the discussions here...

For the HTML language server I understand it basically just needs attributes, but for all other tooling like my WASM idea we need the other data such as the DOM properties, I'd like to see Justin's list become the spec if possible. BTW I appreciate you starting this issue, I'm excited to see where it ends up :D