libuv bindings in stdlib

[graydon]

The standard library will require very carefully arranged bindings to libuv (https://github.com/joyent/libuv), the async IO library we're going to be using. The current rust IO system is synchronous and stdio-based, which is not really ideal long term.

[robarnold]

Seems like this is not high-pri but good in the long term. Assigning to myself if you don't mind.

[graydon]

Its presence (or absence) relates pretty heavily to the set of abstractions we present to users for IO and concurrency. IOW a concurrency system is only as good as the IO subsystem that services it. So I'd like to get it sorted out sooner rather than later. But if you want to dig in, feel free.

[robarnold]

I have a WIP on my fork if you'd like to check it out for some initial feedback.

[brson]

The aio code we have had since summer was bitrotted so I removed it a few weeks ago (I apologize robarnold - I should have discussed it with you).

In 7476a39 I checked in the beginnings of a different approach. What I would like to do is produce a low-level Rust API that closely mirrors the C uv API, with similar performance characteristics, with minimal runtime C++ code. So the user starts an event loop which calls event handler functions, just like uv.

To do so I propose going through the libuv test suite and duplicating each test case in Rust until we have a complete implementation. At that point we can write some networking libraries. I think this is a good approach because writing these bindings is going to be a major test of both our task system and FFI so we need rock solid tests.

Pretty early you run into the issue of dealing with native callbacks, and resolving it is going to require some significant changes to the runtime scheduling code.

This is what I propose:

• Separate rust_scheduler into rust_domain and rust_sched_thread. rust_domain is 'the scheduler' and rust_sched_thread manages the per-thread resources like the C stack and context switching.
• Instead of the kernel having a fixed set of schedulers, it now has a dynamic set of domains which each have a fixed number of threads.
• Implement std::task::spawn_domain which creates a new domain with some specified number of threads (in the future this can be refined with other types of domain).
• Calling uv::run spawns 2 single-threaded domains: an event loop and an event handler, then waits for the event loop to exit. The event loop will spend all its time blocked in C code so must be in its own domain and the handler will need to respond very quickly to events so shouldn't have to deal with other tasks being scheduled ahead of it.
• The event loop handles the native events and synchronously dispatches them to the event handler task which then presents them to the user as a Rust callback. There are several options for how to implement this and I think over time there will be opportunities to optimize, with the ultimate optimization being a mechanism to actually call Rust code from C code, eliminating the event handler task.
• We define all of our uv handles with one extra field: rustfn, a pointer to a Rust function. When the user registers a callback, we put that callback in rustfn and register our own native function to handle the dispatch back to Rust code.

It gets more vague from here as I've only scratched the surface of the libuv API, but I believe this lays out a promising path.

I'm still open to somebody else implementing it as I don't expect to get back to libuv until sometime after 0.1.

[brson]

We have task::spawn_sched now.

[graydon]

While this is not perfect yet, the uv bindings certainly exist now, no? Can we close this and open new bugs for further issues we encounter with the uv bindings?

[brson]

Sure.