[#84867] [Ruby trunk Bug#14357] thread_safe tests suite segfaults — v.ondruch@...
Issue #14357 has been reported by vo.x (Vit Ondruch).
11 messages
2018/01/15
[#85364] Re: [Ruby trunk Bug#14357] thread_safe tests suite segfaults
— Eric Wong <normalperson@...>
2018/02/03
[email protected] wrote:
[#85368] Re: [Ruby trunk Bug#14357] thread_safe tests suite segfaults
— Eric Wong <normalperson@...>
2018/02/03
Eric Wong wrote:
[#85442] Re: [Ruby trunk Bug#14357] thread_safe tests suite segfaults
— Eric Wong <normalperson@...>
2018/02/06
Eric Wong <[email protected]> wrote:
[#85451] Re: [Ruby trunk Bug#14357] thread_safe tests suite segfaults
— Vladimir Makarov <vmakarov@...>
2018/02/06
On 02/06/2018 05:00 AM, Eric Wong wrote:
[#85455] Re: [Ruby trunk Bug#14357] thread_safe tests suite segfaults
— Eric Wong <normalperson@...>
2018/02/06
Vladimir Makarov <[email protected]> wrote:
[#84874] [Ruby trunk Bug#14360] Regression CSV#open method for writing from Ruby 2.4.3 to 2.5.0 — shevegen@...
Issue #14360 has been updated by shevegen (Robert A. Heiler).
3 messages
2018/01/15
[#84980] [Ruby trunk Feature#13618][Assigned] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — hsbt@...
Issue #13618 has been updated by hsbt (Hiroshi SHIBATA).
10 messages
2018/01/23
[#85012] Re: [Ruby trunk Feature#13618][Assigned] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
— Eric Wong <normalperson@...>
2018/01/23
[email protected] wrote:
[#85081] Re: [Ruby trunk Feature#13618][Assigned] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
— Eric Wong <normalperson@...>
2018/01/24
Eric Wong <[email protected]> wrote:
[#85082] Re: [Ruby trunk Feature#13618][Assigned] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
— Eric Wong <normalperson@...>
2018/01/24
> Thinking about this even more; I don't think it's possible to
[#85088] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — danieldasilvaferreira@...
Issue #13618 has been updated by dsferreira (Daniel Ferreira).
3 messages
2018/01/25
[#85107] [Ruby trunk Misc#14222] Mutex.lock is not safe inside signal handler: what is? — eregontp@...
Issue #14222 has been updated by Eregon (Benoit Daloze).
3 messages
2018/01/25
[#85136] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — Eric Wong <normalperson@...>
[email protected] wrote:
3 messages
2018/01/26
[ruby-core:85144] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
From:
danieldasilvaferreira@...
Date:
2018-01-27 03:45:26 UTC
List:
ruby-core #85144
Issue #13618 has been updated by dsferreira (Daniel Ferreira).
normalperson (Eric Wong) wrote:
> I'm not sure what you're talking about. I suppose nothing is reliable
Let me try to explain what I think about the async subject in ruby land using a different story:
For me there is ruby core and there is ruby.
I'm a ruby kind of guy like most of ruby developers.
I like to use ruby and I like to use it as it is given to us by ruby core.
I prefer to build my own tools in top of ruby core rather than using external libraries/gems.
That with the assumption that ruby core will not break backwards compatibility.
If there is something I really dislike is to fix broken code due to dependency issues.
Ruby core is the rock solid foundation I rely upon for the developments I design and implement.
I started in ruby land with rails like most of us.
As years passed by I went more and more to other territories.
So I believe my story it is a very common story:
The ruby developer that starts at the very high level with rails.
With time becomes progressively more and more familiar with the low level concepts of programming.
Gets to understand the underlying concepts behind the frameworks and starts to grasp at last the ruby essence.
And here I am now speaking with you guys.
Ruby core. The lower level by excellence.
It is fascinating to go through discussions like this one.
The craft of ruby landscape for the future.
Technically speaking I'm learning a lot but I'm not prepared yet to give my contribution at that level.
The contribution I believe I can give is this view I'm speaking about.
The daily user that sometimes struggle to find the right paths for the problems in hands.
---
Ruby developers like my self (I imagine there will be more that feel this way) are very much impacted by the opinions of ruby core team members.
Specially top team members like Koichi.
We can call it the teacher - student dichotomy.
When Koichi referring to threads functionality in ruby land writes and says:
> "But most (many? some? a few?) of ruby programmer (including me) can not write correct code I believe."
I do listen. People listen.
(Koichi sentence here is just an handy reference example (sorry Koichi), from the many I have read throughout this many years and many of those comments are here, embedded in redmine issues).
These sentences have a very big impact.
I as a programmer aim to write and develop correct code.
If there is an area that I do not feel comfortable with then I study it, play with it but that is it.
I will not put my job and my company in jeopardy just to show some cool stuff to the team.
Ruby programmer not ruby core hacker remember?
How many ruby developers develop a http server or know the internals of at least one?
(Just as an example of different levels of developer seniority.)
Unicorn or passenger or thin or puma... are black boxes for the most of us.
And yes there are bugs and our applications are impacted by them.
That is the ecosystem and it is good like that. It will not change.
Somehow there are people that feel happy playing in dangerous zones like threads and fibers
(See previous Koichi reference. We know you have a "slightly" different opinion).
Us, mere mortals, just would like to be able to do our daily work at least without compromising.
Although I would like to use libraries to play with my actors without worrying to much I can't.
Knowing that they are dangerous zones tells me I must worry still.
So, why not use Akka and live happy ever after?
In Akka land everyone happily uses actors.
I never heard any reference telling people to be careful about a given issue.
Maybe the issues exist but what you read is that Akka is the solution for all your problems in async world.
I don't want to use Akka but I know that ruby is losing developers every day because of situations like this one I'm referring here.
Ruby desperately needs to resolve once and for all this situation.
The key word for me here is a clear message that could say with confidence:
"Ruby is rock solid for async because..."
If we don't succeed to pass this message to the world of programming ruby will slowly be replaced by other languages.
Parallelism and concurrency and async will be everywhere in the future.
I took the decision to express this thoughts in this conversation because I love ruby and I want to help ruby become better.
In my opinion:
We need to create the foundations for a post ruby 3 future in ruby land where async is the standard for the many and not the exception for the few.
That is my vision.
Many Thanks,
Daniel
----------------------------------------
Feature #13618: [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
https://bugs.ruby-lang.org/issues/13618#change-69880
* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: normalperson (Eric Wong)
* Target version:
----------------------------------------
```
auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
Implement automatic Fiber yield and resume when running
rb_wait_for_single_fd and rb_waitpid.
The Ruby API changes for Fiber are named after existing Thread
methods.
main Ruby API:
Fiber#start -> enable auto-scheduling and run Fiber until it
automatically yields (due to EAGAIN/EWOULDBLOCK)
The following behave like their Thread counterparts:
Fiber.start - Fiber.new + Fiber#start (prelude.rb)
Fiber#join - run internal scheduler until Fiber is terminated
Fiber#value - ditto
Fiber#run - like Fiber#start (prelude.rb)
Right now, it takes over rb_wait_for_single_fd() and
rb_waitpid() function if the running Fiber is auto-enabled
(cont.c::rb_fiber_auto_sched_p)
Changes to existing functions are minimal.
New files (all new structs and relations should be documented):
iom.h - internal API for the rest of RubyVM (incomplete?)
iom_internal.h - internal header for iom_(select|epoll|kqueue).h
iom_epoll.h - epoll-specific pieces
iom_kqueue.h - kqueue-specific pieces
iom_select.h - select-specific pieces
iom_pingable_common.h - common code for iom_(epoll|kqueue).h
iom_common.h - common footer for iom_(select|epoll|kqueue).h
Changes to existing data structures:
rb_thread_t.afrunq - list of fibers to auto-resume
rb_vm_t.iom - Ruby I/O Manager (rb_iom_t) :)
Besides rb_iom_t, all the new structs are stack-only and relies
extensively on ccan/list for branch-less, O(1) insert/delete.
As usual, understanding the data structures first should help
you understand the code.
Right now, I reuse some static functions in thread.c,
so thread.c includes iom_(select|epoll|kqueue).h
TODO:
Hijack other blocking functions (IO.select, ...)
I am using "double" for timeout since it is more convenient for
arithmetic like parts of thread.c. Most platforms have good FP,
I think. Also, all "blocking" functions (rb_iom_wait*) will
have timeout support.
./configure gains a new --with-iom=(select|epoll|kqueue) switch
libkqueue:
libkqueue support is incomplete; corner cases are not handled well:
1) multiple fibers waiting on the same FD
2) waiting for both read and write events on the same FD
Bugfixes to libkqueue may be necessary to support all corner cases.
Supporting these corner cases for native kqueue was challenging,
even. See comments on iom_kqueue.h and iom_epoll.h for
nuances.
Limitations
Test script I used to download a file from my server:
----8<---
require 'net/http'
require 'uri'
require 'digest/sha1'
require 'fiber'
url = 'http://80x24.org/git-i-forgot-to-pack/objects/pack/pack-97b25a76c03b489d4cbbd85b12d0e1ad28717e55.idx'
uri = URI(url)
use_ssl = "https" == uri.scheme
fibs = 10.times.map do
Fiber.start do
cur = Fiber.current.object_id
# XXX getaddrinfo() and connect() are blocking
# XXX resolv/replace + connect_nonblock
Net::HTTP.start(uri.host, uri.port, use_ssl: use_ssl) do |http|
req = Net::HTTP::Get.new(uri)
http.request(req) do |res|
dig = Digest::SHA1.new
res.read_body do |buf|
dig.update(buf)
#warn "#{cur} #{buf.bytesize}\n"
end
warn "#{cur} #{dig.hexdigest}\n"
end
end
warn "done\n"
:done
end
end
warn "joining #{Time.now}\n"
fibs[-1].join(4)
warn "joined #{Time.now}\n"
all = fibs.dup
warn "1 joined, wait for the rest\n"
until fibs.empty?
fibs.each(&:join)
fibs.keep_if(&:alive?)
warn fibs.inspect
end
p all.map(&:value)
Fiber.new do
puts 'HI'
end.run.join
```
---Files--------------------------------
0001-auto-fiber-schedule-for-rb_wait_for_single_fd-and-rb.patch (82.8 KB)
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:[email protected]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>
