Refactor smoke UI automation into separate package

[jasongin]

This change divides the smoke project into two packages:

• The existing test/smoke package now contains only the actual test cases plus the code for setting up and running those tests via mocha.
• A new package at test/automation contains the UI automation "driver" and per-component automation modules. The smoke package depends on this one via a local filesystem reference.

Besides enforcing a clearer separation between those two parts, this also makes it possible to build a reusable npm package for VS Code UI automation. VS Code might never support broad consumption of this package, and there is no immediate plan to publish it to the global npm registry. My goal is to make it easier for some first-party extensions (VS Live Share and others) to run UI tests in VS Code, and consume updates to the test automation package when testing updated releases of VS Code. There is no implication or expectation that the automation package or any part of the VS Code DOM would avoid breaking changes or follow semver rules. But by re-using the automation modules we can reduce the amount of effort required to adapt to the changes.

I verified the existing VS Code smoke tests are still passing with these changes. The process for building the tests is slightly different due to having two packages where there was one; I have tried to make it as simple as possible, and I updated the smoke README. Let me know if you'd like something done differently.

@joaomoreno @Tyriar

[Tyriar]

FYI @DonJayamanne @SounD120

[Tyriar]

Might be better to have "precompile": "yarn --cwd ../automation compile"?

[jasongin]

Actually I intended this to install packages there also. (If packages are already installed, yarn is very quick to skip that step.) Otherwise you would have to go manually do a yarn install in the automation directory first.

[Tyriar]

When you run yarn in the root folder it will run for a bunch of sub-folders, can we not hook into that same mechanism? It's being done for test/smoke currently.

[jasongin]

OK, I'll look into it.

[jasongin]

Done. I added yarnInstall('test/automation') to the root postinstall.js script. And in this package.json I removed the preinstall script and updated the compile script to also compile the automation package. I think that simplifies the workflow when you might be changing both test/automation and test/smoke at the same time.

[joaomoreno]

@Tyriar, since you've been deep into this lately, do you mind taking this?

[Tyriar]

@joaomoreno sure, I already looked over the whole thing and it looks pretty good, just those comments above and testing it myself left.

[Tyriar]

Watch now doesn't work quite right because of how tsc rewrites to the terminal, you'll want to use the --preserveWatchOutput flag to avoid this

I did this

[Tyriar]

Just the yarn thing remaining, then good to merge. Anything needed after this for your team to start using it, or is that it?

[jasongin]

We don't need anything else right now. We've already been using this automation library for nearly a year now to test VS Live Share. It was just a tedious manual process to extract from the vscode repo in a usuable form. Now that will be simple, so we can take updates more frequently.

I had planned to contribute some small changes to the automation library to make multi-window automation easier. For Live Share we automate scenarios with host in one window and guest in another. Currently we have to access some private members of the Code class to make that work. I'll make a separate PR for that.

[Tyriar]

Thanks for finishing up the yarn stuff 👍