[#86787] [Ruby trunk Feature#14723] [WIP] sleepy GC — ko1@...
Issue #14723 has been updated by ko1 (Koichi Sasada).
13 messages
2018/05/01
[#86790] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/01
[email protected] wrote:
[#86791] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/01
On 2018/05/01 12:18, Eric Wong wrote:
[#86792] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/01
Koichi Sasada <[email protected]> wrote:
[#86793] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/01
On 2018/05/01 12:47, Eric Wong wrote:
[#86794] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/01
Koichi Sasada <[email protected]> wrote:
[#86814] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/02
[#86815] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/02
Koichi Sasada <[email protected]> wrote:
[#86816] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/02
On 2018/05/02 11:49, Eric Wong wrote:
[#86847] [Ruby trunk Bug#14732] CGI.unescape returns different instance between Ruby 2.3 and 2.4 — me@...
Issue #14732 has been reported by jnchito (Junichi Ito).
3 messages
2018/05/02
[#86860] [Ruby trunk Feature#14723] [WIP] sleepy GC — sam.saffron@...
Issue #14723 has been updated by sam.saffron (Sam Saffron).
6 messages
2018/05/03
[#86862] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/03
[email protected] wrote:
[#86935] [Ruby trunk Bug#14742] Deadlock when autoloading different constants in the same file from multiple threads — elkenny@...
Issue #14742 has been reported by eugeneius (Eugene Kenny).
5 messages
2018/05/08
[#87030] [Ruby trunk Feature#14757] [PATCH] thread_pthread.c: enable thread caceh by default — normalperson@...
Issue #14757 has been reported by normalperson (Eric Wong).
4 messages
2018/05/15
[#87093] [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase — ko1@...
Issue #14767 has been updated by ko1 (Koichi Sasada).
3 messages
2018/05/17
[#87095] [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase — ko1@...
Issue #14767 has been updated by ko1 (Koichi Sasada).
9 messages
2018/05/17
[#87096] Re: [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase
— Eric Wong <normalperson@...>
2018/05/17
[email protected] wrote:
[#87166] Re: [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase
— Eric Wong <normalperson@...>
2018/05/18
Eric Wong <[email protected]> wrote:
[#87486] Re: [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase
— Eric Wong <normalperson@...>
2018/06/13
I wrote:
[ruby-core:87209] [Ruby trunk Feature#14759] [PATCH] set M_ARENA_MAX for glibc malloc
From:
dennisb55@...
Date:
2018-05-21 05:19:18 UTC
List:
ruby-core #87209
Issue #14759 has been updated by bluz71 (Dennis B). Mike, Yusuke script is still interesting for the datum that a Ruby script with `MALLOC_ARENA_MAX=2` consumed **more** memory than a case using the default arena count (usually 32 on 4 core machines). Here are my results of Yusuke's script on my 4-core machine (Intel i5-4590 quad-core, 16GB RAM, Linux Mint 18.3 with kernel 4.15.0). ~~~ % time ruby frag2.rb VmRSS: 1,238,108 kB real 0m38.792s % time MALLOC_ARENA_MAX=2 ruby frag2.rb VmRSS: 1,561,624 kB real 0m39.002s % time MALLOC_ARENA_MAX=4 ruby frag2.rb VmRSS: 1,516,216 kB real 0m36.614s % time MALLOC_ARENA_MAX=32 ruby frag2.rb VmRSS: 1,218,180 kB real 0m36.857s ~~~ This is perplexing. Clearly Ruby should not be changing defaults until we understand results like this. Here are jemalloc results (3.6.0 first and 5.0.1 second): ~~~ % time LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so ruby frag2.rb VmRSS: 933,328 kB real 1m33.660s % time LD_PRELOAD=/home/bluz71/.linuxbrew/Cellar/jemalloc/5.0.1/lib/libjemalloc.so ruby frag2.rb VmRSS: 1,613,252 kB real 0m27.530s ~~~ jemalloc 3.6.0 is extremely slow but with very low RSS. jemalloc 5.0.1 is very fast (much faster than glibc) but also has the highest RSS. Ruby can not be linked against a system supplied jemalloc since different jemalloc versions are extremely different; jemalloc 3.6 and 5.0 are basically different allocators that share the same name. But we need to always remember that long lived Ruby processes on Linux have a very bad memory fragmentation as seen here:  What can the Ruby maintainers do? I am less certain now what they should do than a week ago. But I still want to see the memory fragmentation issue reduced for default Ruby. ---------------------------------------- Feature #14759: [PATCH] set M_ARENA_MAX for glibc malloc https://bugs.ruby-lang.org/issues/14759#change-72199 * Author: normalperson (Eric Wong) * Status: Open * Priority: Normal * Assignee: * Target version: ---------------------------------------- Not everybody benefits from jemalloc and the extra download+install time is not always worth it. Lets make the user experience for glibc malloc users better, too. Personally, I prefer using M_ARENA_MAX=1 (via MALLOC_ARENA_MAX env) myself, but there is currently a performance penalty for that. gc.c (Init_GC): set M_ARENA_MAX=2 for glibc malloc glibc malloc creates too many arenas and leads to fragmentation. Given the existence of the GVL, clamping to two arenas seems to be a reasonable trade-off for performance and memory usage. Some users (including myself for several years, now) prefer only one arena, now, so continue to respect users' wishes when MALLOC_ARENA_MAX is set. Thanks to Mike Perham for the reminder [ruby-core:86843] This doesn't seem to conflict with jemalloc, so it should be safe for all glibc-using systems. ---Files-------------------------------- 0001-gc.c-Init_GC-set-M_ARENA_MAX-2-for-glibc-malloc.patch (1.46 KB) -- https://bugs.ruby-lang.org/ Unsubscribe: <mailto:[email protected]?subject=unsubscribe> <http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>
