[#84280] [Ruby trunk Bug#14181] hangs or deadlocks from waitpid, threads, and trapping SIGCHLD — nobu@...
Issue #14181 has been updated by nobu (Nobuyoshi Nakada).
3 messages
2017/12/15
[#84398] [Ruby trunk Bug#14220] WEBrick changes - failures on MSWIN, MinGW — Greg.mpls@...
Issue #14220 has been reported by MSP-Greg (Greg L).
3 messages
2017/12/22
[#84472] Re: [ruby-dev:50394] [Ruby trunk Bug#14240] warn four special variables: $; $, $/ $\ — Eric Wong <normalperson@...>
Shouldn't English posts be on ruby-core instead of ruby-dev?
3 messages
2017/12/26
[ruby-core:84149] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
From:
samuel@...
Date:
2017-12-10 22:30:13 UTC
List:
ruby-core #84149
Issue #13618 has been updated by ioquatix (Samuel Williams).
I think that the work being done here is great. However I feel that this PR requires far more scrutiny than it's receiving.
It's worth considering that nio4r and friends took several years to stabilise and there is a huge amount of hard earned knowledge embedded in those gems, e.g.
> I am using "double" for timeout since it is more convenient for arithmetic like parts of thread.c. Most platforms have good FP, I think.
e.g. https://github.com/socketry/nio4r/issues/140
I think it's a great idea to have non-blocking evented IO. However, it's not as simple as making read/write non-blocking. How about DNS lookups? Filesystem access? The benefit of a library based approach as I proposed is that these limitations can be clearly part of the contract of a specific library, and people can make different libraries to suit their needs, but making it part of core Ruby is a slippery slope. If anything, it would be better to depend on an established solution for this, so that cases like using the system DNS resolver are handled correctly (e.g. libuv). Otherwise, this is a HUGE addition to the surface area of the ruby interpreter.
----------------------------------------
Feature #13618: [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
https://bugs.ruby-lang.org/issues/13618#change-68253
* Author: normalperson (Eric Wong)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
----------------------------------------
```
auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
Implement automatic Fiber yield and resume when running
rb_wait_for_single_fd and rb_waitpid.
The Ruby API changes for Fiber are named after existing Thread
methods.
main Ruby API:
Fiber#start -> enable auto-scheduling and run Fiber until it
automatically yields (due to EAGAIN/EWOULDBLOCK)
The following behave like their Thread counterparts:
Fiber.start - Fiber.new + Fiber#start (prelude.rb)
Fiber#join - run internal scheduler until Fiber is terminated
Fiber#value - ditto
Fiber#run - like Fiber#start (prelude.rb)
Right now, it takes over rb_wait_for_single_fd() and
rb_waitpid() function if the running Fiber is auto-enabled
(cont.c::rb_fiber_auto_sched_p)
Changes to existing functions are minimal.
New files (all new structs and relations should be documented):
iom.h - internal API for the rest of RubyVM (incomplete?)
iom_internal.h - internal header for iom_(select|epoll|kqueue).h
iom_epoll.h - epoll-specific pieces
iom_kqueue.h - kqueue-specific pieces
iom_select.h - select-specific pieces
iom_pingable_common.h - common code for iom_(epoll|kqueue).h
iom_common.h - common footer for iom_(select|epoll|kqueue).h
Changes to existing data structures:
rb_thread_t.afrunq - list of fibers to auto-resume
rb_vm_t.iom - Ruby I/O Manager (rb_iom_t) :)
Besides rb_iom_t, all the new structs are stack-only and relies
extensively on ccan/list for branch-less, O(1) insert/delete.
As usual, understanding the data structures first should help
you understand the code.
Right now, I reuse some static functions in thread.c,
so thread.c includes iom_(select|epoll|kqueue).h
TODO:
Hijack other blocking functions (IO.select, ...)
I am using "double" for timeout since it is more convenient for
arithmetic like parts of thread.c. Most platforms have good FP,
I think. Also, all "blocking" functions (rb_iom_wait*) will
have timeout support.
./configure gains a new --with-iom=(select|epoll|kqueue) switch
libkqueue:
libkqueue support is incomplete; corner cases are not handled well:
1) multiple fibers waiting on the same FD
2) waiting for both read and write events on the same FD
Bugfixes to libkqueue may be necessary to support all corner cases.
Supporting these corner cases for native kqueue was challenging,
even. See comments on iom_kqueue.h and iom_epoll.h for
nuances.
Limitations
Test script I used to download a file from my server:
----8<---
require 'net/http'
require 'uri'
require 'digest/sha1'
require 'fiber'
url = 'http://80x24.org/git-i-forgot-to-pack/objects/pack/pack-97b25a76c03b489d4cbbd85b12d0e1ad28717e55.idx'
uri = URI(url)
use_ssl = "https" == uri.scheme
fibs = 10.times.map do
Fiber.start do
cur = Fiber.current.object_id
# XXX getaddrinfo() and connect() are blocking
# XXX resolv/replace + connect_nonblock
Net::HTTP.start(uri.host, uri.port, use_ssl: use_ssl) do |http|
req = Net::HTTP::Get.new(uri)
http.request(req) do |res|
dig = Digest::SHA1.new
res.read_body do |buf|
dig.update(buf)
#warn "#{cur} #{buf.bytesize}\n"
end
warn "#{cur} #{dig.hexdigest}\n"
end
end
warn "done\n"
:done
end
end
warn "joining #{Time.now}\n"
fibs[-1].join(4)
warn "joined #{Time.now}\n"
all = fibs.dup
warn "1 joined, wait for the rest\n"
until fibs.empty?
fibs.each(&:join)
fibs.keep_if(&:alive?)
warn fibs.inspect
end
p all.map(&:value)
Fiber.new do
puts 'HI'
end.run.join
```
---Files--------------------------------
0001-auto-fiber-schedule-for-rb_wait_for_single_fd-and-rb.patch (82.8 KB)
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:[email protected]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>
