[#88240] [Ruby trunk Feature#14759] [PATCH] set M_ARENA_MAX for glibc malloc — sam.saffron@...
Issue #14759 has been updated by sam.saffron (Sam Saffron).
3 messages
2018/08/01
[#88251] Re: [ruby-alerts:8236] failure alert on trunk@P895 (NG (r64134)) — Eric Wong <normalperson@...>
[email protected] wrote:
3 messages
2018/08/01
[#88305] [Ruby trunk Bug#14968] [PATCH] io.c: make all pipes nonblocking by default — normalperson@...
Issue #14968 has been reported by normalperson (Eric Wong).
4 messages
2018/08/06
[#88331] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — samuel@...
Issue #13618 has been updated by ioquatix (Samuel Williams).
3 messages
2018/08/08
[#88342] [Ruby trunk Feature#14955] [PATCH] gc.c: use MADV_FREE to release most of the heap page body — ko1@...
Issue #14955 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/08
[#88433] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — ko1@...
SXNzdWUgIzEzNjE4IGhhcyBiZWVuIHVwZGF0ZWQgYnkga28xIChLb2ljaGkgU2FzYWRhKS4KCgpX
4 messages
2018/08/10
[#88467] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
— Eric Wong <normalperson@...>
2018/08/14
a28xQGF0ZG90Lm5ldCB3cm90ZToKPiBJc3N1ZSAjMTM2MTggaGFzIGJlZW4gdXBkYXRlZCBieSBr
[#88475] [Ruby trunk Misc#14937] [PATCH] thread_pthread: lazy-spawn timer-thread only on contention — ko1@...
Issue #14937 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/14
[#88491] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test — Eric Wong <normalperson@...>
[email protected] wrote:
5 messages
2018/08/15
[#88495] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test
— Takashi Kokubun <takashikkbn@...>
2018/08/15
SSBzZWUuIFBsZWFzZSByZW1vdmUgdGhlIHRlc3QgaWYgdGhlIHRlc3QgaXMgdW5uZWNlc3Nhcnku
[#88496] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test
— Eric Wong <normalperson@...>
2018/08/15
Takashi Kokubun <[email protected]> wrote:
[#88523] [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed — eregontp@...
Issue #14999 has been updated by Eregon (Benoit Daloze).
4 messages
2018/08/17
[#88524] Re: [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed
— Eric Wong <normalperson@...>
2018/08/18
[email protected] wrote:
[#88549] [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed — eregontp@...
Issue #14999 has been updated by Eregon (Benoit Daloze).
3 messages
2018/08/19
[#88676] [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations — ko1@...
Issue #15014 has been updated by ko1 (Koichi Sasada).
5 messages
2018/08/27
[#88677] Re: [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations
— Eric Wong <normalperson@...>
2018/08/27
[email protected] wrote:
[#88678] Re: [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations
— Koichi Sasada <ko1@...>
2018/08/27
On 2018/08/27 16:16, Eric Wong wrote:
[#88716] Re: [ruby-dev:43715] [Ruby 1.9 - Bug #595] Fiber ignores ensure clause — Eric Wong <normalperson@...>
Koichi Sasada wrote:
4 messages
2018/08/29
[#88723] [Ruby trunk Bug#15041] [PATCH] cont.c: set th->root_fiber to current fiber at fork — ko1@...
Issue #15041 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/29
[#88767] [Ruby trunk Bug#15050] GC after forking with fibers crashes — ko1@...
Issue #15050 has been updated by ko1 (Koichi Sasada).
7 messages
2018/08/31
[#88769] Re: [Ruby trunk Bug#15050] GC after forking with fibers crashes
— Eric Wong <normalperson@...>
2018/08/31
Koichi Sasada <[email protected]> wrote:
[#88770] Re: [Ruby trunk Bug#15050] GC after forking with fibers crashes
— Koichi Sasada <ko1@...>
2018/08/31
[#88773] Re: [Ruby trunk Bug#15050] GC after forking with fibers crashes
— Eric Wong <normalperson@...>
2018/08/31
Koichi Sasada <[email protected]> wrote:
[#88774] Re: [ruby-alerts:8955] failure alert on trunk@P895 (NG (r64594)) — Eric Wong <normalperson@...>
[email protected] wrote:
3 messages
2018/08/31
[ruby-core:88478] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
From:
danieldasilvaferreira@...
Date:
2018-08-14 09:00:15 UTC
List:
ruby-core #88478
Issue #13618 has been updated by dsferreira (Daniel Ferreira). [email protected] wrote: > I am thinking of adding preemption support to this feature for compatibility with 1.8 non-preemptive vs preemptive. coroutines are non-preemptive. threads are preemptive. Are we talking about having the two behaviours in this new feature? > So this made me think of "Thread::Coro" What is the logic behind "Coro"? > Other ideas: Thread::CSP or Thread::Sequential (probably too long) Does it mean we will have the CSP algebraic operators available? Reading through this conversation it feels we are dealing with a feature with a lot of concepts incorporated into it. Can we get a resume of all the functionality we expect to have? Some code examples would be great. Or even feature comparison with other languages. I believe we must do that kind of documentation to show to the community in a clear way the new ruby async possibilities. I'm willing to help in planning and developing it. Many thanks. ---------------------------------------- Feature #13618: [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid https://bugs.ruby-lang.org/issues/13618#change-73542 * Author: normalperson (Eric Wong) * Status: Assigned * Priority: Normal * Assignee: normalperson (Eric Wong) * Target version: ---------------------------------------- ``` auto fiber schedule for rb_wait_for_single_fd and rb_waitpid Implement automatic Fiber yield and resume when running rb_wait_for_single_fd and rb_waitpid. The Ruby API changes for Fiber are named after existing Thread methods. main Ruby API: Fiber#start -> enable auto-scheduling and run Fiber until it automatically yields (due to EAGAIN/EWOULDBLOCK) The following behave like their Thread counterparts: Fiber.start - Fiber.new + Fiber#start (prelude.rb) Fiber#join - run internal scheduler until Fiber is terminated Fiber#value - ditto Fiber#run - like Fiber#start (prelude.rb) Right now, it takes over rb_wait_for_single_fd() and rb_waitpid() function if the running Fiber is auto-enabled (cont.c::rb_fiber_auto_sched_p) Changes to existing functions are minimal. New files (all new structs and relations should be documented): iom.h - internal API for the rest of RubyVM (incomplete?) iom_internal.h - internal header for iom_(select|epoll|kqueue).h iom_epoll.h - epoll-specific pieces iom_kqueue.h - kqueue-specific pieces iom_select.h - select-specific pieces iom_pingable_common.h - common code for iom_(epoll|kqueue).h iom_common.h - common footer for iom_(select|epoll|kqueue).h Changes to existing data structures: rb_thread_t.afrunq - list of fibers to auto-resume rb_vm_t.iom - Ruby I/O Manager (rb_iom_t) :) Besides rb_iom_t, all the new structs are stack-only and relies extensively on ccan/list for branch-less, O(1) insert/delete. As usual, understanding the data structures first should help you understand the code. Right now, I reuse some static functions in thread.c, so thread.c includes iom_(select|epoll|kqueue).h TODO: Hijack other blocking functions (IO.select, ...) I am using "double" for timeout since it is more convenient for arithmetic like parts of thread.c. Most platforms have good FP, I think. Also, all "blocking" functions (rb_iom_wait*) will have timeout support. ./configure gains a new --with-iom=(select|epoll|kqueue) switch libkqueue: libkqueue support is incomplete; corner cases are not handled well: 1) multiple fibers waiting on the same FD 2) waiting for both read and write events on the same FD Bugfixes to libkqueue may be necessary to support all corner cases. Supporting these corner cases for native kqueue was challenging, even. See comments on iom_kqueue.h and iom_epoll.h for nuances. Limitations Test script I used to download a file from my server: ----8<--- require 'net/http' require 'uri' require 'digest/sha1' require 'fiber' url = 'http://80x24.org/git-i-forgot-to-pack/objects/pack/pack-97b25a76c03b489d4cbbd85b12d0e1ad28717e55.idx' uri = URI(url) use_ssl = "https" == uri.scheme fibs = 10.times.map do Fiber.start do cur = Fiber.current.object_id # XXX getaddrinfo() and connect() are blocking # XXX resolv/replace + connect_nonblock Net::HTTP.start(uri.host, uri.port, use_ssl: use_ssl) do |http| req = Net::HTTP::Get.new(uri) http.request(req) do |res| dig = Digest::SHA1.new res.read_body do |buf| dig.update(buf) #warn "#{cur} #{buf.bytesize}\n" end warn "#{cur} #{dig.hexdigest}\n" end end warn "done\n" :done end end warn "joining #{Time.now}\n" fibs[-1].join(4) warn "joined #{Time.now}\n" all = fibs.dup warn "1 joined, wait for the rest\n" until fibs.empty? fibs.each(&:join) fibs.keep_if(&:alive?) warn fibs.inspect end p all.map(&:value) Fiber.new do puts 'HI' end.run.join ``` ---Files-------------------------------- 0001-auto-fiber-schedule-for-rb_wait_for_single_fd-and-rb.patch (82.8 KB) -- https://bugs.ruby-lang.org/ Unsubscribe: <mailto:[email protected]?subject=unsubscribe> <http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>
