Hot exit support for custom editors

[mjbvz]

Discussion of how to implement hot exit for custom editors (#77131)

Background

With hot exit users can quit VS Code with unsaved file changes and know that those files (in their edited state) will be restored when they open VS Code again.

VS Code can properly restore the user's previous file state even in cases such as the following:

• User edits a file and then exits VS Code without saving
• In a different program, the user then edits the same file and saves it to disk
• Then the user reopens VS Code

In that case, VS Code correctly restores the original file with edits. VS Code implements hot exit by continuously persisting edited files to a backup folder, and then checking that backup folder when restoring VS Code.

If there is an error that would prevent VS Code from hot exiting, such as if the backup fails, VS Code will alert the user and require the the non-backed up files be either saved to disk or have their changes lost.

Hot exit + Custom editors

We would like to support hot exit for custom editors as well. From the user's point of view, hot exit of a custom editor should behave the same way it does for normal text editors.

Goals

• Allow extensions to support hot exit for both text and binary based custom editors
• Don't require transferring binary file data from extensions to VS Code
  • Make it possible to implement hot exit for larger files
• Support hot exit for custom editors that are not on disk or that are backed by multiple resources
• Fail gracefully (no data loss)

Big questions

• What part of the file state does VS Code back up? What part of the state do extension back up?
  • Edits?
  • File contents?
• Where is the backup stored?
  • Who cleans up old backups?
• How do extensions know that a backup exists and restore the editors from this?
• How does VS Code know that a given editor is backed up? (i.e. we can hot exit)
• When/how are extension notified that they need to back up? For example, if auto save is enabled, we may not need to backup since the file will be saved shortly anyways

How we implement backup also influences the rest of the custom editor API. For example, does it still make sense to have the concept of edits in the API, or could this be delegated to extensions?

[mjbvz]

Review of some offline discussions

Can't VS Code just backup everything?

Yes, so long as edits are json serializable and backing up a custom editors only requires backing up a single file. To implement this, VS Code would simply write the edits + the current disk file to a backup location

However, this falls apart for some interesting cases:

• If edits cannot easily be serialized to json and restore.
• If a custom editor is not backed up by a on-disk resource (or is backed by multiple resources)
• Large files. VS Code would always have to copy around file contents which may not be very efficient?

So in practice, not really.

Do we need to tell extensions where to write backup data?

No. In fact, giving extensions a path to write backups is problematic because we then have to tell extensions where to restore their backups from as well

Extensions can use the workspace specific storage folder instead.

Can we reasonably expect extensions to implement hot exit on their own if we provide only documentation or a very small set of api hooks?

No. We can't assume that extensions will properly implement hot exit as there are lots of fun edge cases (such as the file on disk changing between sessions) that they will likely not consider.

Making hot exit an opt-in feature that advanced custom editors can implement may be a good idea.

Do we need any API at all for hot exit?

Strictly speaking, no. Extensions could mostly implement hot exit using the existing custom editor api. However we would likely need some kind of api for extensions to tell us what state backup is in (ongoing, failed, successful)

There are a also some points where minimal api could help reduce code duplication:

• Knowing if backups are required or not (such as when auto save is on)?
• Knowing when to backup
• ...?

What is actually saved during a backup? Do we save off the edits + original file? The file with the edits applied?

Maybe it's best to leave this to extensions as well.

During restore, all VS Code really needs to do is tell an extension to open a custom editor. That extension can then figure out if it has a backup or not and decide how to restore itself.

Some extensions may restore by simply loading the edited file contents, other by loading the original file and then re-applying the edits to it.

Do edits need to be json serializable?

Not if VS Code never needs to write them to disk or restore them across sessions.

We could keep edit objects on the extension host side and just give the VS Code process simple handles to them.

Is there any advantage then to having the concept of edits in the API?

Perhaps.

Edits give VS Code knowledge of the the edit stack. This enable features such as:

• Disabling undo/redo based on the current edit stack.
• Not requiring an explicit revert hook
• Letting VS Code control the dirty state of files.

Disadvantages:

• It may be difficult for some extensions to implement this properly
• Not clear at first how to implement aggregated edits (but I think it is possible)
• If I edit a file back into a un-edited state, there is no way for an extension to signal that the file is no longer dirty

[bpasero]

On backup & auto save: if auto save is enabled and configured to save after short delay (anything with or below 1s - which is our default) we do not backup at all for content changes. Upon shutdown, we save any dirty editor automatically if not yet saved (e.g. the user closing VSCode before auto save had a chance to kick in).

[mjbvz]

A few sketches of what a minimal api could look like:

Optional prepareForHotExit method

Add a new, optional method on the WebviewCustomEditorEditingDelegate class that backed

interface WebviewCustomEditorEditingDelegate<EditType> {

	/**
	 * Back up the resource in its current state
	 * 
	 * This would be invoked by VS Code when a new edit comes in
	 *  
	 * @param resource Resource to back up.
	 * @param cancellation Signals that the current backup is no longer needed. For example, if the user makes an edit and then quickly saves the file.
	 * 
	 * @return Thenable signaling that the editor is prepared to hot exit or not.
	 */
	prepareForHotExit?(resource: Uri, cancellation: CancellationToken): Thenable<boolean>;
}

Optional event that signals if an editor can hot exit or not

Add a onHotExitChange event that lets extension signal to VS Code that the editor can hot exit. We would leave it up to extension to fire this whenever the editor state changes.

interface WebviewCustomEditorEditingDelegate<EditType> {
	/**
	 * Event that signals to VS Code that the current editor supports hot exit.
	 */
	readonly onHotExitChange?: Event<{
		canHotExit: Thenable<boolean>;
	}>;
}

[mjbvz]

@bpasero For web, you mentioned that we have to be able to synchronously check if we can hot exit. In order to avoid data loss, I'm currently thinking that internally VS Code will need to track three hot exit states for each editor:

• CanHotExit — We can safely hot exit.
• CannotHotExit — We cannot hot exit.
• Pending — We are determining if we can hot exit or not.

On desktop, we would delay exiting while all Pending hot exit states are resolved. On web, we would assume that Pending means that we cannot hot exit.

In the api sketches, CanHotExit is signaled by returning true, CannotHotExit by returning false or by not implementing prepareForHotExit, while an unresolved promise would be treated as Pending

Do you think that this approach would work for web? Anything important that I am overlooking?

[bpasero]

I think the requirements are different depending on the platform:

• web: we simply need to know sync if any document is dirty and not backed up to show a dialog to prevent the unload. I think this can be done with 2 states, either editor is backed up or not
• desktop: here, we actually trigger the backup automatically on shutdown if we detect that an editor is dirty and hot exit is enabled (it can also be turned off by the user). not sure how that would work in your model where we rely only on events. Do we then wait for N seconds and check again if the resources have been backed up? with a more explicit backup() method, the workbench could drive this on shutdown.

[mjbvz]

My thinking was we would tell extensions to back up the resource after every edit (possibly with a denounce). That means that we don't have to perform extra work on shutdown (besides possibly waiting for any long running backups to complete)

---

For example, here's how the prepareForHotExit method could work:

• After an edit is made in a custom editor, VS Code invokes prepareForHotExit. At this point, VS Code would also consider the resource as not being backed up (it would be in the Pending state I outlined above)
• The extension's prepareForHotExit method is invoked. The extension starts backing up the resource.
• If another edit happens before prepareForHotExit completes, VS Code cancels the original prepareForHotExit call and invokes prepareForHotExit again
• When VS Code gets a result back from prepareForHotExit, it stores the current hot exit state (CanHotExit or CannotHotExit) until the next edit comes in.

On desktop, when quitting VS Code with hot exit enabled we would try to resolve all pending calls to prepareForHotExit. We would not need to invoke prepareForHotExit at this point (unless we have debounced it). If prepareForHotExit returns false for any known resources (or if we hit some timeout), we do not hot exit

On web, when quitting VS Code with hot exit enabled we would first synchronously check all known hot exit states. If there are any pending calls to prepareForHotExit or any previous calls to prepareForHotExit have returned false, we do not hot exit

---

The other approach we discussed is having an event that extensions use to tell VS Code that they can hot exit. However we need to be careful to avoid potential data loss here. For example, consider the following scenario

• The user makes an edit
• The extension backs up the file and fires a canHotExitEvent(true) event.
• VS Code receives this event and updates some internal state that notes that the resource is backed up
• The user then makes another edit
• At this point, the extension should likely immediately fire a canHotExitEvent(false) event since it needs to back up the resource again
• However before VS Code core receives this event, the user now quits
• On desktop, shutdown tasks start. During these, VS Code sees that the resource is backed up (using its internal state from the previous event) and exits before the the extension actually has time to finish its backup

[bpasero]

@mjbvz giving some progress of my current work, didn't have time yet to comment on your proposal.

I am thinking to make the working copy service capable of a long running backup into the extension. Here is what I am sketching up:

interface IWorkingCopyBackup {
	meta?: object;
	content?: ITextSnapshot;
}

interface IWorkingCopy {
   backup(): Promise<IWorkingCopyBackup>;
}

The reason why I am going away from a backup():Promise<void> and hasBackup(): boolean model is that I really want to make sure the IBackupFileService is used in every case. The reason are the following advantages:

• any resource of any backup will be opened as editor on startup
• any workspace with a backup will be opened into a window automatically on startup (e.g. this is useful when the user starts VSCode explicitly on a specific workspace and not the one that had dirty files before)

In other words, making everyone use IBackupFileService gives the most consistent experience.

Now, I kept content in IWorkingCopyBackup strictly optional because we do not want to move any textual data from extension to workbench. Still, you might want to put some meta into the backup to figure out how to restore the editor later, but even that is optional.

IWorkingCopy#backup() will be called automatically every 1s after the content has changed. You are free to communicate this to the extension as appropriate. Here is the flow I have in mind:

Backup Flow:

• user opens a custom editor
• user types into the custom editor
• content change events are emitted
• IWorkingCopy#backup() is called after 1s
• the custom editor asks the extension to backup
• once the extension acknowledges, the custom editor returns the promise from the IWorkingCopy#backup() call
• that's it, from this moment the workbench will happily close even with a dirty custom editor, given we have a backup

Restore Flow:

• user opens VSCode
• we find backups
• we open each backup as editor
• custom editor is asked to open an input
• custom editor activates the extension
• extension is responsible for restoring the backup

As you can see, this model makes backups mandatory and not optional, I think we should try for this goal for the best possible experience

Working Copy Resource Identity
We will have revisit how resources are used for custom editors and their working copies. Since the IBackupFileService is all resource based, we cannot simply use the same file resource for a custom editor that might be used for a normal text editor already - the backups would overwrite each other.

Instead, I am thinking this: the custom editor could add some identifier to the file resource to identity the viewType it is associated with from the override logic. This would also allow the custom editor to properly restore the right viewType when asked to open the resource again. The flow would be:

• user opens a file (file:///User/bpasero/cat.abc)
• custom editor checks for override
• custom editor finds a viewType: myExtensionCustomEditor
• custom editor modifies the resource to identify this and makes that the resource of the editor and working copy (file:///User/bpasero/cat.abc?viewType=myExtensionCustomEditor)
• once the workbench restarts, that resource will be asked to open (openEditor({ resource: file:///User/bpasero/cat.abc?viewType=myExtensionCustomEditor}))
• custom editor can check if there is any viewType query param to figure out the extension to activate

Thoughts?

PS: not yet pushed, will do Tuesday (out tomorrow)

[mjbvz]

That sounds like an interesting idea. However I wonder if custom editors really need to use VS Code's backup service at all. The only information that a custom editor would backup on the VS Code side would be the file path and its editor type. Both of those can be serialized and restored on the editor input.

I need to think through your proposal more. Let's discuss it further next week!

---

Also, I have a draft PR that implements prepareForHotExit: #88948 This PR will help me test how well can extension can actually implement hot exit with a simple API and understand if we've overlooked any major issues

[bpasero]

However I wonder if custom editors really need to use VS Code's backup service at all. The only information that a custom editor would backup on the VS Code side would be the file path and its editor type.

@mjbvz There is one case which is tricky to get right where the backup file service would help you. Imagine this scenario:

• user opens custom editors and makes them dirty in a workspace
• user closes VSCode
• user runs from the command line "code <some other workspace>"
• this will open 2 windows: the one the user asked for and the other to restore the dirty custom editors (this is done from code on the electron-main side that knows that there are backups for a certain workspace)

I know the happy path is easy to get right by relying on the editors being restored from the UI state, but there are cases where this will not work. This is why I think having everyone use the same service is a good idea.

[bpasero]

I have pushed IWorkingCopy#backup() as described. We can discuss if it can be optional or not, currently it is mandatory.