[#116016] [Ruby master Bug#20150] Memory leak in grapheme clusters — "peterzhu2118 (Peter Zhu) via ruby-core" <ruby-core@...>
Issue #20150 has been reported by peterzhu2118 (Peter Zhu).
7 messages
2024/01/04
[#116382] [Ruby master Feature#20205] Enable `frozen_string_literal` by default — "byroot (Jean Boussier) via ruby-core" <ruby-core@...>
Issue #20205 has been reported by byroot (Jean Boussier).
77 messages
2024/01/23
[ruby-core:116397] [Ruby master Feature#20205] Enable `frozen_string_literal` by default
From:
"byroot (Jean Boussier) via ruby-core" <ruby-core@...>
Date:
2024-01-24 09:44:43 UTC
List:
ruby-core #116397
Issue #20205 has been updated by byroot (Jean Boussier). > I think we should evaluate the value of # frozen_string_literal: true before making it the default. So it's not 100% reliable because I ran it locally rather than on a benchmarking server, as evidenced by some strange effect on benchmarks that don't normally deal with strings (e.g. `setivar_object`), but here are the results: ``` mutable: ruby 3.4.0dev (2024-01-24T08:24:16Z disable-frozen-str.. a39d5eae1e) [arm64-darwin23] frozen: ruby 3.4.0dev (2024-01-24T08:24:16Z disable-frozen-str.. bb0cee8dab) [arm64-darwin23] -------------- ------------ ---------- ----------- ---------- -------------- -------------- bench mutable (ms) stddev (%) frozen (ms) stddev (%) frozen 1st itr mutable/frozen activerecord 32.9 4.0 32.7 5.3 1.06 1.01 chunky-png 578.0 1.1 555.2 1.0 1.01 1.04 erubi-rails 1124.2 2.4 1158.7 1.7 0.93 0.97 hexapdf 1703.9 2.7 1670.2 2.2 1.04 1.02 liquid-c 34.3 5.7 34.1 5.0 1.02 1.01 liquid-compile 39.8 4.2 38.0 5.3 0.99 1.05 liquid-render 97.2 3.4 97.7 3.5 1.01 0.99 lobsters 637.8 5.2 614.9 4.2 1.11 1.04 mail 82.7 3.7 82.1 3.7 1.00 1.01 psych-load 1500.8 0.8 1487.6 0.5 0.99 1.01 railsbench 1116.3 2.1 1099.8 1.6 1.02 1.01 rubocop 112.0 3.8 111.2 3.7 1.03 1.01 ruby-lsp 79.5 2.6 80.3 2.4 0.92 0.99 sequel 36.4 4.4 36.5 3.3 0.95 1.00 binarytrees 238.4 1.9 238.8 1.7 0.99 1.00 blurhash 281.9 1.4 282.5 1.7 0.99 1.00 erubi 166.1 2.0 171.5 2.5 0.97 0.97 etanni 198.7 2.7 200.5 1.8 1.04 0.99 fannkuchredux 2065.3 0.7 2078.5 0.5 0.99 0.99 fluentd 1192.0 1.0 1202.7 0.7 1.03 0.99 graphql 2323.6 0.7 2340.5 0.5 0.98 0.99 graphql-native 385.3 1.5 384.2 2.1 1.00 1.00 lee 739.3 1.8 750.1 1.2 0.98 0.99 matmul 1494.2 0.8 1497.8 0.9 1.01 1.00 nbody 71.1 2.3 71.3 5.1 1.00 1.00 nqueens 169.8 1.6 167.6 2.0 1.00 1.01 optcarrot 4202.7 0.6 4216.8 0.5 1.00 1.00 rack 56.0 3.3 53.9 3.9 1.06 1.04 ruby-json 1976.3 0.9 1992.4 0.3 1.00 0.99 rubykon 6971.3 0.8 7053.4 0.4 1.00 0.99 sudoku 1836.1 0.4 1836.3 0.3 1.00 1.00 tinygql 441.4 1.0 447.4 1.2 0.96 0.99 30k_ifelse 1459.0 8.6 1429.1 4.6 0.96 1.02 30k_methods 3331.8 3.7 3264.0 1.1 1.03 1.02 cfunc_itself 81.5 1.8 81.2 2.9 1.02 1.00 fib 187.0 1.6 188.8 1.5 0.94 0.99 getivar 65.1 2.2 65.7 2.4 1.01 0.99 keyword_args 141.6 2.3 140.9 2.1 1.00 1.01 respond_to 184.9 1.9 185.7 1.1 0.96 1.00 setivar 39.2 2.9 39.2 2.4 1.03 1.00 setivar_object 72.7 2.5 77.5 2.1 0.95 0.94 setivar_young 72.5 1.9 77.5 0.9 0.93 0.94 str_concat 69.7 2.5 69.9 1.9 1.02 1.00 throw 14.9 5.0 14.8 4.5 1.04 1.01 -------------- ------------ ---------- ----------- ---------- -------------- -------------- Legend: - frozen 1st itr: ratio of mutable/frozen time for the first benchmarking iteration. - mutable/frozen: ratio of mutable/frozen time. Higher is better for frozen. Above 1 represents a speedup. ``` I used the following patch to disable frozen string literals globally: https://github.com/ruby/ruby/compare/master...Shopify:ruby:disable-frozen-string-literal, and ran the suite with `ruby run_benchmarks.rb --chruby 'mutable::head-mutable-strings;frozen::head'`. Most of these benchmarks don't really deal with string, but one that I think is the most close to reality and to Ruby bread and butter is `lobsters` and it seem to be quite positive. I also didn't enable YJIT, I suspect the difference would be bigger if I did, as the extra GC pressure would be relatively bigger. I'll see about using the yjit-perf benchmark server to run these more scientifically. I also want to note that I have to explictly freeze `RUBY_DESCRIPTION` in my patch, otherwise it would cause a Ractor issue, which is another argument for frozen strings, as they ease the necessary work necessary to make code Ractor compatible. ---------------------------------------- Feature #20205: Enable `frozen_string_literal` by default https://bugs.ruby-lang.org/issues/20205#change-106419 * Author: byroot (Jean Boussier) * Status: Open * Priority: Normal ---------------------------------------- ### Context The `frozen_string_literal: true` pragma was introduced in Ruby 2.3, and as far as I'm aware the plan was initially to make it the default for Ruby 3.0, but this plan was abandoned because it would be too much of a breaking change without any real further notice. According to Matz, he still wishes to enable `frozen_string_literal` by default in the future, but a reasonable migration plan is required. The main issue is backward compatibility, flipping the switch immediately would break a lot of code, so there must be some deprecation period. The usual the path forward for this kind of change is to emit deprecation warnings one of multiple versions in advance. One example of that was the Ruby 2.7 keyword argument deprecation. It was quite verbose, and some users were initially annoyed, but I think the community pulled through it and I don't seem to hear much about it anymore. So for frozen string literals, the first step would be to start warning when a string that would be frozen in the future is mutated. ### Deprecation Warning Implementation I implemented a quick proof of concept with @etienne in https://github.com/Shopify/ruby/pull/549 In short: - Files with `# frozen_string_literal: true` or `# frozen_string_literal: false` don't change in behavior at all. - Files with no `# frozen_string_literal` comment are compiled to use `putchilledstring` opcode instead of regular `putstring`. - This opcode mark the string with a user flag, when these strings are mutated, a warning is issued. Currently the proof of concept issue the warning at the mutation location, which in some case can make locating where the string was allocated a bit hard. But it is possible to improve it so the message also include the location at which the literal string was allocated, and learning from the keyword argument warning experience, we can record which warnings were already issued to avoid spamming users with duplicated warnings. As currently implemented, there is almost no overhead. If we modify the implementation to record the literal location, we'd incur a small memory overhead for each literal string in a file without an explicit `frozen_string_literal` pragma. But I believe we could do it in a way that has no overhead if `Warning[:deprecated] = false`. ### Timeline The migration would happen in 3 steps, each step can potentially last multiple releases. e.g. `R0` could be `3.4`, `R1` be `3.7` and `R2` be `4.0`. I don't have a strong opinion on the pace. - Release `R0`: introduce the deprecation warning (only if deprecation warnings enabled). - Release `R1`: make the deprecation warning show up regardless of verbosity level. - Release `R2`: make string literals frozen by default. ### Impact Given that `rubocop` is quite popular in the community and it has enforced the usage of `# frozen_string_literal: true` for years now, I suspect a large part of the actively maintained codebases in the wild wouldn't see any warnings. And with recent versions of `minitest` enabling deprecation warnings by default (and [potentially RSpec too](https://github.com/rspec/rspec-core/issues/2867)), the few that didn't migrate will likely be made compatible quickly. The real problem of course are the less actively developed libraries and applications. For such cases, any codebase can remain compatible by setting `RUBYOPT="--disable=frozen_string_literal"`, and so even after `R2` release. The flag would never be removed any legacy codebase can continue upgrading Ruby without changing a single line of cod by just flipping this flag. ### Workflow for library maintainers As a library maintainer, fixing the deprecation warnings can be as simple as prepending `# frozen_string_literal: false` at the top of all their source files, and this will keep working forever. Alternatively they can of course make their code compatible with frozen string literals. Code that is frozen string literal compatible doesn't need to explicitly declare it. Only code that need it turned of need to do so. ### Workflow for application owners For application owners, the workflow is the same than for libraries. However if they depend on a gem that hasn't updated, or that they can't upgrade it, they can run their application with `RUBYOPT="--disable=frozen_string_literal"` and it will keep working forever. Any user running into an incompatibility issue can set `RUBYOPT="--disable=frozen_string_literal"` forever, even in `4.x`, the only thing changing is the default value. And any application for which all dependencies have been made fully frozen string literal compatible can set `RUBYOPT="--enable=frozen_string_literal"` and start immediately removing magic comment from their codebase. -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- [email protected] To unsubscribe send an email to [email protected] ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/
