Explore using tree sitter for syntax highlighting

Goals of the explorations:

Get Aiday's exploration up and running again: [Exploration] Tree-sitter tokenization exploration (Fixes #161256) #161479
Come up with ideas for a contribution model
Confirm performance findings from [Exploration] Tree-sitter tokenization exploration (Fixes #161256) #161479 and understand incremental highlighting

Henning Dieterichs (@hediet)

Some notes on incremental parsing:

Per Can web-tree-sitter actually do async parsing? tree-sitter/tree-sitter#2118, it looks like we can set a timeout so that each call to parse times-out, with the parsing progress preserved for the next call to parse.
I've confirmed that this is the case; however, it appears to be broken in the latest version of web-tree-sitter. Going back to 0.22.2 works (latest is 0.22.5).
There does not appear to be any significant time penalty for setting a timeout.

Opened tree-sitter/tree-sitter#3341 for the above issue.

After some discussion, we've decided to first understand the performance of using tree-sitter to parse files. #213565 contains that work.

Peng Lyu (@rebornix)

We also need new internal editor API to write a more optimal parse function, which Peng Lyu (@rebornix) may help with.

looking forward to tree-sitter for golang

going to ON DECK

going to ON DECK

Nice,

github.com/microsoft/vscode/milestone/27

The Issues assigned to the On Deck milestone under consideration for the upcoming milestones.
Our goal is to always have less than 100 Issues on deck.

742 open issues 😬

Hello,

I was looking for news about tree sitter support in vscode, and couldn't find anything newer than what's here and is from more than 6 months ago.

So, concretely, are there any performance improvements with tree sitter compared to TextMate? More/better supported languages?

Thanks!

Rémi Marche (@ecstrema)

Rémi Marche (@ecstrema) it's still On Deck, meaning under consideration for the upcoming milestones.

7 Months ago, there were less issues on deck than now, although their goal was to keep the issues there < 100 apparently. It is now sitting at ~800 open issues.

The priority for vscode (or any Microsoft product) seems to be better Copilot/AI integration right now.

The upside of tree sitter highlighting is that it's way more convenient to write them compared to textmate, as they tend to get quite large and tedious to write from hand.

Textmate is regex based has to have the ability to restart at any line.
(see 12.2 Language Rules)

Note that the regular expressions are matched against only a single line of the document at a time. That means it is not possible to use a pattern that matches multiple lines.

Defining actual scopes in Textmate is this nearly impossible, and a approximation at best. Inserting line breaks in some places will break some highlighting, but you tend to not notice them as the grammars are written in ways they expect you to write your code.

Tree sitter has actual syntax 'rules' and scopes where different rules apply, so this not an issue.

So you can expect better and more consistent support for languages.
You can look at the comments above or the related issues for performance insights, but this seems like a non-issue.

Moved to backlog 😭

Navigation Menu

Repository navigation

Explore using tree sitter for syntax highlighting #210475

95 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Explore using tree sitter for syntax highlighting #210475

Description

Activity

alexr00 commented on May 3, 2024

alexr00 commented on May 6, 2024

alexr00 commented on May 28, 2024

alexr00 commented on May 28, 2024

95 remaining items

pinkyo commented on Jul 16, 2025

heartacker commented on Jul 16, 2025

Frulfump commented on Jul 16, 2025

ecstrema commented on Feb 5, 2026

Plixo2 commented on Feb 5, 2026

MulverineX commented on Jun 16, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions