Stable build?

[jedahan]

Might be worth tracking what would need to happen for tiny to build on stable. Personal context is the (musl-based) distribution I use only packages stable and rustup running nightly doesn't seem to work.

Happy to at the very least just track non-stable dependencies and features to be able to check back in.

[osa1]

Have you checked the releases page? We already provide a statically-linked (musl) executable.

I just removed a few feature attributes that were not really needed (either unused, or I was able to replace them easily). Remaining nightly features are:

• or_patterns: Could be easily removed, at the cost of some minor code duplication.
• ptr_offset_from: Used to implement SplitWhitespaceIndices. Shouldn't be too hard to implement this without the feature. It turns out ptr_offset_from is stable now.
• test: Used to implement benchmarks in TUI. No idea how to implement benchmarks without this.

Only difficulty seems to be (3). Interestingly I just tried running the benchmarks and I see this:

test msg_area::line::tests::bench_rendered_height ... bench:           0 ns/iter (+/- 0)
test trie::benchs::bench_trie_build               ... bench:           0 ns/iter (+/- 0)
test trie::benchs::bench_trie_list_all            ... bench:           0 ns/iter (+/- 0)
test trie::benchs::bench_trie_lookup              ... bench:           0 ns/iter (+/- 0)

It seems like they're not really run. I have no idea why, would be good to investigate.

[trevarj]

Regarding the benchmarkes:

Chatted with @osa1 about this and he found that the three trie benchmarks aren't running because the path is wrong. It's looking for a dict file called "american", but on my system it's "american-english". Changing that allows the benches to run.
For the msg_area bench, it doesn't work correctly because of the line caching that I added. I fixed this by forcing the recalculation each time:

        b.iter(|| {
            line.force_recalculation();
            line.rendered_height(1);
        });

Since the goal here is to get tiny running on Rust stable, I then started porting the benchmarks to use criterion.rs (also looking at bencher (stable port of libtest). I don't really like this because it will require making trie a public module and then changing the visibility of Trie. What are your thoughts on this @osa1 ?

[trevarj]

Criterion results for fun:

     Running target/release/deps/tui_benches-f20761e04089ffe4
Gnuplot not found, using plotters backend
trie_build              time:   [31.689 ms 31.830 ms 31.985 ms]                       
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

trie_list_all           time:   [33.653 ms 33.744 ms 33.840 ms]                          
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) high mild
  1 (1.00%) high severe

trie_lookup             time:   [22.811 us 22.856 us 22.904 us]                         
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low mild
  7 (7.00%) high mild
  1 (1.00%) high severe

[trevarj]

After the bench changes, and fixing the easy or_patterns, the most difficult remaining issue is the use of offset_from in utils.rs. The good news is that this will be available in stable 1.47.

[osa1]

After the bench changes, and fixing the easy or_patterns, the most difficult remaining issue is the use of offset_from in utils.rs. The good news is that this will be available in stable 1.47.

Indeed, that's why I removed it in 90e4cc1.

Tbh, I'm not too keen on moving to stable. But I could be convinced. I'll write more on this later.

[osa1]

Thinking about this more, perhaps it makes sense to move to stable.

The reason why I was hesitant at first is because I've been using nightly for this project for many years and I never had any stability problems with it, except one or two times, in more than 4 years. So nightly has always been stable enough for our use case.

Also at some point I had about 5-6 unstable features enabled. It was simply too difficult to write Rust without unstable features (e.g. non-lexical lifetimes was essential). This is no longer the case, as this issue proved. We only have two unstable features left, both of which can be replaced without much inconvenience.

So it seems like for our use case there isn't too much difference between the nightly and stable.

I personally don't see why some users use their distro's Rust package (rather than rustup), but if this will enable people to use tiny or even contribute to it, that's a good thing.

---

Re: criterion vs. bencher, why does bencher require making trie stuff public but not criterion? I think we should use the one that is more widely used as that'll probably be more stable and have better API, more examples etc. which seems to be criterion.

[trevarj]

Re: criterion vs. bencher, why does bencher require making trie stuff public but not criterion? I think we should use the one that is more widely used as that'll probably be more stable and have better API, more examples etc. which seems to be criterion.

I think both require this because the benchmarks need to be outside of src/ in a crate. In other words, I don't believe inner benching of crates works at the moment. Criterion seems to be more widely used/supported and has more features (stats, charts).

[osa1]

Hmm... I wonder if having so many features that we don't need is a good thing. Perhaps bencher is a better alternative for us as it doesn't have any dependencies, and we don't use black_box (which bencher is missing I think).

[jedahan]

Just for some background info of why I cannot use rustup - it does not work with musl, so we have to patch cargo. We attempt to put these patches upstream but that takes time. Otherwise yeah I’d use rustup for the other benefits.

Another more general reason people might not use rustup is if they are not developing on rust, just want to install packages. Similar to why one would not install nvm but would like to run nodejs-based software.

[osa1]

I briefly looked at replacing the nightly's test crate this weekend. The main problem is one of the benchmarks is implemented for an internal data structure used very deep in the TUI code. It'd be weird to make the module path, from root to the benchmarked type, public, just to be able to implement a benchmark.

(I also discovered that that benchmark got more than 10x slower in the mantime, but that's another issue ...)

I think we may be able to implement a better TUI benchmark that doesn't rely on internals. Just one idea: create a TUI but with termbox in test mode. Add large text with many lines (maybe lipsum.txt). Start with some reasonable screen size, maybe 80x50, and then in a loop resize it by making it 1x1 smaller in each step (call draw() after each resize) until the screen size is 1x1, then do it in reverse back to the original size.

[osa1]

I ported the benchmarks to bencher crate in #245.

We still have to wait for the next stable, to be able to use ptr_offset_from.

[osa1]

All unstable features removed, we just need to wait for the next stable release now.