Add combine-based benchmark that caused a large performance regression

[sdroege]

See rust-lang/rust#67454

[sdroege]

This will probably time out on travis until the PR above is merged, it's taking quite a few minutes with latest nightly and after that PR is merged it will take about 10s.

[Mark-Simulacrum]

We'll probably want to remove a few of the combine calls to get the post-PR runtime down to ~2-3 seconds at most; we try to keep the benchmarks in this repository very short as we run them lots of times while benchmarking (9 times minimum by default).

[sdroege]

combine itself needs around 9s to build on my machine. The benchmark crate alone needs around 200ms with 1.40 and more than 20 minutes with current nightly.

[Mark-Simulacrum]

The benchmark crate alone needs around 200ms with 1.40 and more than 20 minutes with current nightly.

Hm, so you originally said it takes around 10s after the PR -- does that mean there's still a regression vs. 1.40, or am I misinterpreting?

[sdroege]

Hm, so you originally said it takes around 10s after the PR -- does that mean there's still a regression vs. 1.40, or am I misinterpreting?

Until rust-lang/rust#67454 is merged it (the benchmark crate itself, not its dependencies) will take quite a few minutes. With 1.40 and nightly after that is merged, combine itself takes about 9s on my machine and the benchmark crate itself 200ms on top of that.

[Mark-Simulacrum]

Okay, so in that case, we're actually going to want to bump the runtime of the benchmark itself up a bit I guess? We want it in the 600-1500ms range I suspect.

[rylev]

This is pretty stale so I'm going to close for now. Feel free to reopen if you'd like to continue to work on it!