[#86787] [Ruby trunk Feature#14723] [WIP] sleepy GC — ko1@...
Issue #14723 has been updated by ko1 (Koichi Sasada).
13 messages
2018/05/01
[#86790] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/01
[email protected] wrote:
[#86791] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/01
On 2018/05/01 12:18, Eric Wong wrote:
[#86792] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/01
Koichi Sasada <[email protected]> wrote:
[#86793] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/01
On 2018/05/01 12:47, Eric Wong wrote:
[#86794] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/01
Koichi Sasada <[email protected]> wrote:
[#86814] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/02
[#86815] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/02
Koichi Sasada <[email protected]> wrote:
[#86816] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Koichi Sasada <ko1@...>
2018/05/02
On 2018/05/02 11:49, Eric Wong wrote:
[#86847] [Ruby trunk Bug#14732] CGI.unescape returns different instance between Ruby 2.3 and 2.4 — me@...
Issue #14732 has been reported by jnchito (Junichi Ito).
3 messages
2018/05/02
[#86860] [Ruby trunk Feature#14723] [WIP] sleepy GC — sam.saffron@...
Issue #14723 has been updated by sam.saffron (Sam Saffron).
6 messages
2018/05/03
[#86862] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC
— Eric Wong <normalperson@...>
2018/05/03
[email protected] wrote:
[#86935] [Ruby trunk Bug#14742] Deadlock when autoloading different constants in the same file from multiple threads — elkenny@...
Issue #14742 has been reported by eugeneius (Eugene Kenny).
5 messages
2018/05/08
[#87030] [Ruby trunk Feature#14757] [PATCH] thread_pthread.c: enable thread caceh by default — normalperson@...
Issue #14757 has been reported by normalperson (Eric Wong).
4 messages
2018/05/15
[#87093] [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase — ko1@...
Issue #14767 has been updated by ko1 (Koichi Sasada).
3 messages
2018/05/17
[#87095] [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase — ko1@...
Issue #14767 has been updated by ko1 (Koichi Sasada).
9 messages
2018/05/17
[#87096] Re: [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase
— Eric Wong <normalperson@...>
2018/05/17
[email protected] wrote:
[#87166] Re: [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase
— Eric Wong <normalperson@...>
2018/05/18
Eric Wong <[email protected]> wrote:
[#87486] Re: [Ruby trunk Feature#14767] [PATCH] gc.c: use monotonic counters for objspace_malloc_increase
— Eric Wong <normalperson@...>
2018/06/13
I wrote:
[ruby-core:87199] [Ruby trunk Feature#14759] [PATCH] set M_ARENA_MAX for glibc malloc
From:
mperham@...
Date:
2018-05-20 17:58:30 UTC
List:
ruby-core #87199
Issue #14759 has been updated by mperham (Mike Perham).
Yusuke, your script doesn't create any memory fragmentation, it throws away everything after 1600 and reads the exact same amount of data each time. I don't believe this is how Rails apps behave; they fragment over time. My script creates random sized data and holds onto 10% of the data to create "holes" in the heap and fragment memory quickly. I believe this better represents normal app conditions. I've edited your script slightly to randomly keep some data; it better matches the results I posted earlier. I think changing the IO to read random sizes would also exhibit worse memory:
~~~
$ time MALLOC_ARENA_MAX=2 /ruby/2.5.1/bin/ruby frag2.rb
VmRSS: 1620356 kB
real 1m20.755s
user 0m38.057s
sys 1m2.881s
$ time /ruby/2.5.1/bin/ruby frag2.rb
VmRSS: 1857284 kB
real 1m19.642s
user 0m36.645s
sys 1m4.480s
~~~
~~~
$ more frag2.rb
THREAD_COUNT = (ARGV[0] || "10").to_i
File.write("/tmp/tmp.txt", "x" * 1024 * 64)
srand(1234)
Threads = []
Save = []
THREAD_COUNT.times do
Threads << Thread.new do
a = []
100_000.times do
a = open("/tmp/tmp.txt") {|f| f.read }
Save << a if rand(100_000) < 1600
end
end
end
Threads.each {|th| th.join }
GC.start
IO.foreach("/proc/#{$$}/status") do |line|
print line if line =~ /VmRSS/
end if RUBY_PLATFORM =~ /linux/
~~~
----------------------------------------
Feature #14759: [PATCH] set M_ARENA_MAX for glibc malloc
https://bugs.ruby-lang.org/issues/14759#change-72188
* Author: normalperson (Eric Wong)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
----------------------------------------
Not everybody benefits from jemalloc and the extra download+install
time is not always worth it. Lets make the user experience for
glibc malloc users better, too.
Personally, I prefer using M_ARENA_MAX=1 (via MALLOC_ARENA_MAX
env) myself, but there is currently a performance penalty for
that.
gc.c (Init_GC): set M_ARENA_MAX=2 for glibc malloc
glibc malloc creates too many arenas and leads to fragmentation.
Given the existence of the GVL, clamping to two arenas seems
to be a reasonable trade-off for performance and memory usage.
Some users (including myself for several years, now) prefer only
one arena, now, so continue to respect users' wishes when
MALLOC_ARENA_MAX is set.
Thanks to Mike Perham for the reminder [ruby-core:86843]
This doesn't seem to conflict with jemalloc, so it should be safe
for all glibc-using systems.
---Files--------------------------------
0001-gc.c-Init_GC-set-M_ARENA_MAX-2-for-glibc-malloc.patch (1.46 KB)
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:[email protected]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>
