i686 OSX cycle time has regressed 20%

[alexcrichton]

From https://travis-ci.org/rust-lang/rust/jobs/210149964 to https://travis-ci.org/rust-lang/rust/jobs/214694967 we've regressed cycle time by 20% unfortunately:

run Testing libtest stage2 (i686-apple-darwin -> i686-apple-darwin)                                 :   -14.09%    26.86 =>    23.08
Testing libtest stage2 (i686-apple-darwin -> i686-apple-darwin)                                     :    -2.94%    20.77 =>    20.16
Building stage0 tool compiletest (i686-apple-darwin)                                                :    -2.27%    25.96 =>    25.37
Building stage0 tool tidy (i686-apple-darwin)                                                       :    -2.57%     7.40 =>     7.21
Building stage0 tool error_index_generator (i686-apple-darwin)                                      :     6.47%     6.18 =>     6.58
Documenting standalone (i686-apple-darwin)                                                          :     3.87%    14.73 =>    15.30
 suite=compile-fail mode=compile-fail (i686-apple-darwin -> i686-apple-darwin)                      :     0.18%   405.72 =>   406.45
 suite=run-pass-valgrind mode=run-pass-valgrind (i686-apple-darwin -> i686-apple-darwin)            :    40.10%     2.17 =>     3.04
 suite=codegen-units mode=codegen-units (i686-apple-darwin -> i686-apple-darwin)                    :    14.12%     6.54 =>     7.46
Building stage0 test artifacts (i686-apple-darwin -> i686-apple-darwin)                             :     6.08%    18.10 =>    19.20
 suite=parse-fail mode=parse-fail (i686-apple-darwin -> i686-apple-darwin)                          :    21.59%     5.82 =>     7.08
 suite=mir-opt mode=mir-opt (i686-apple-darwin -> i686-apple-darwin)                                :    53.21%     2.57 =>     3.93
Linkcheck (i686-apple-darwin)                                                                       :    13.69%    10.62 =>    12.07
Building stage0 tool linkchecker (i686-apple-darwin)                                                :    40.21%     3.83 =>     5.37
 suite=run-fail-fulldeps mode=run-fail (i686-apple-darwin -> i686-apple-darwin)                     :    78.84%     2.20 =>     3.93
 suite=ui-fulldeps mode=ui (i686-apple-darwin -> i686-apple-darwin)                                 :   110.35%     1.78 =>     3.74
 suite=codegen mode=codegen (i686-apple-darwin -> i686-apple-darwin)                                :    46.80%     4.36 =>     6.41
Building stage1 test artifacts (i686-apple-darwin -> i686-apple-darwin)                             :    20.34%    22.96 =>    27.63
Documenting stage2 std (i686-apple-darwin)                                                          :    15.55%    32.54 =>    37.60
 suite=ui mode=ui (i686-apple-darwin -> i686-apple-darwin)                                          :    33.21%    15.42 =>    20.54
 suite=run-fail mode=run-fail (i686-apple-darwin -> i686-apple-darwin)                              :    40.74%    25.41 =>    35.76
 suite=incremental mode=incremental (i686-apple-darwin -> i686-apple-darwin)                        :    27.99%    48.75 =>    62.39
Testing error-index stage2                                                                          :    31.21%    46.27 =>    60.71
Building stage1 std artifacts (i686-apple-darwin -> i686-apple-darwin)                              :    19.68%    75.34 =>    90.17
Building stage0 std artifacts (i686-apple-darwin -> i686-apple-darwin)                              :    17.57%    84.93 =>    99.85
 suite=debuginfo mode=debuginfo-lldb (i686-apple-darwin -> i686-apple-darwin)                       :     9.95%   159.37 =>   175.23
Building stage0 tool rustbook (i686-apple-darwin)                                                   :    11.20%   144.23 =>   160.38
 suite=run-make mode=run-make (i686-apple-darwin -> i686-apple-darwin)                              :    23.56%   108.54 =>   134.12
Testing libstd stage2 (i686-apple-darwin -> i686-apple-darwin)                                      :    13.45%   246.35 =>   279.49
 suite=compile-fail-fulldeps mode=compile-fail (i686-apple-darwin -> i686-apple-darwin)             :    65.93%    54.55 =>    90.52
 suite=rustdoc mode=rustdoc (i686-apple-darwin -> i686-apple-darwin)                                :   106.83%    49.19 =>   101.74
Building stage0 compiler artifacts (i686-apple-darwin -> i686-apple-darwin)                         :     5.50%  1063.86 =>  1122.36
 suite=run-pass-fulldeps mode=run-pass (i686-apple-darwin -> i686-apple-darwin)                     :    54.16%   164.53 =>   253.64
Testing librustc stage2 (i686-apple-darwin -> i686-apple-darwin)                                    :    24.13%   400.30 =>   496.88
run Testing librustc stage2 (i686-apple-darwin -> i686-apple-darwin)                                :    25.88%   408.11 =>   513.72
Building LLVM for i686-apple-darwin                                                                 :    32.52%   355.42 =>   471.02
 suite=run-pass mode=run-pass (i686-apple-darwin -> i686-apple-darwin)                              :    15.90%  1006.42 =>  1166.41
Building stage1 compiler artifacts (i686-apple-darwin -> i686-apple-darwin)                         :    18.33%  1416.21 =>  1675.80
run Testing libstd stage2 (i686-apple-darwin -> i686-apple-darwin)                                  :    35.60%   811.16 =>  1099.89
TOTAL                                                                                               :    19.80%  7305.47 =>  8752.23

The list of changes is unfortunately not very illuminating ...

[alexcrichton]

Also just so I don't forget it, the source I used to generate that output is: https://gist.github.com/alexcrichton/40101e6d700cd2dd0f16969807fbb87e

[alexcrichton]

@brson has thoughts on another thread

[ishitatsuyuki]

It seems the test time varies for over 70% difference. Maybe Travis is using multiple types of OSX machines?

Short: https://travis-ci.org/rust-lang/rust/builds/237034153
Long: https://travis-ci.org/rust-lang/rust/builds/237011292

[Mark-Simulacrum]

The long tests you linked are a poor comparison, since that PR hasn't merged yet and may have an underlying bug of some kind; https://travis-ci.org/rust-lang/rust/builds/236851493 is probably a better comparison point.

[jackh726]

Going to close as unactionable. We have better processes today to track performance regressions and this particular issue probably isn't going to get looked at.