Suggestion: Look into porting to Rust and using Tokio

[brucod]

Alex Crichton, from the Rust Core team, gave a great presentation on Concurrency in Rust with Async I/O using Tokio at code::dive 2017. As soon as I watched your Deno talk, I thought about his. It is long, but I highly recommend watching it.

From his talk: Using a simple hello world benchmark, they achieved almost 2mil req/sec - nearly 250k more than the next best implementation and 500k more than Go's fasthttp. Out of the box, Tokio supports 'TCP, UDP, Unix sockets, Named pipes, processes, signals, http, http2, web sockets, ...'

Some observations (Some may be wrong and definitely missing some 😃):

• Mio crate handles all the OS async abstractions for you for each major platform bundled into Tokio. No re-inventing the wheel.
• They don't use callbacks like Node but futures and task-scheduling.
• Actual threadsafe concurrent/multi-threaded async operations and servers. See Rayon as well
• Async/await Rust syntax that might map well to V8/JS's async/await implementation.
• Rust is already poised for WebAssembly. Writing Deno in Rust, developers won't have to know Go as well.
• Rust is young but growing - voted most loved language for third year in a row Stack Overflow's yearly survey
• Tokio's thoughtful design appears to lend to extensible use cases and crates.
• Tokio is written and supported by members of the Rust Core team.
• No garbage collection

Thanks for the cool, new project and your time

Edit: Forgot to include no GC

[arjunyel]

One block is Deno uses Protobufs for communication but they are not officially supported in Rust, There are third party libraries tho.

I think Rust would be a great fit, it seems Mozilla is putting a lot of work into WebAssembly 😄

[FrankFang]

https://github.com/ry/deno/issues/208#issuecomment-395739171
https://github.com/ry/deno/pull/209#issuecomment-395985554

[fluxxu]

I am a huge Rust fan, and I agree with the advantages you listed. But I don't think now is the best time for deno to invest to Futures/Tokio, considering

• Future has some difficult ergonomics problems now.
• Future and task system were just merged into std 2 days ago.
  Add Future and task system to the standard library rust-lang/rust#51263 (comment)
• async/await is not implemented now. I know the design work is almost done. And there is a prototype futures-await. But it takes time to settle all the details.
• Rust 2018 is expected to release in September. I am not saying it will break everything, but some major refactoring is foreseeable.

These are not deal breakers, and Rust core team is resolving them quickly. It's just not sensible to add these complexities to deno now, which itself an experimental project.
I think the better approach is to implement low-level modules using C++ now, and later port them to Rust after Rust's async io story settled.

[matiasinsaurralde]

I've been playing around with Rust and wrote this wrapper of libv8worker2, might not be useful but I wanted to explore some of the language capabilities for this: https://github.com/matiasinsaurralde/rust-v8worker2

[ghost]

The biggest problem is not going to be making the native part of deno fast - there are tons of fast native servers. Where everything is going to be determined is how lightweight the JS wrapper of this server will be. Clearly Node.js completely failed at this, even though their native components are not all that slow.

A fast native server pushing 5 million pipelined hello world HTTP requests per second can be completely butchered by an inefficient JS wrapper, capping out at only a few thousand req/sec (see Node.js) while a well thought out wrapper with minimal amount of dynamic JS resources per request may very well keep up to 20% of the native performance, landing at 1 million req/sec from within JS (see "uws" for Node.js and Japronto for Python cases). The JS wrapping is by far the most perf. sensitive component.

[ghost]

What I would like to see as a highly prioritized item on the agenda is a benchmark of Deno as is, with Golang. Because if Deno cannot properly retain even the perf. of the built-in Golang HTTP module then it will make no difference at all spending a lot of time rewriting the native parts in Rust. We really need to see a benchmark of Deno in action as soon as possible. This should be priority nr. 1 before every other decision really. Otherwise we end up with the same story of Node.js: an overly optimized HTTP parser written in C that does 50 billion req/sec on paper but when put into Node.js caps out at barely 18k req/sec.

[matiasinsaurralde]

@alexhultman I've been running a few benchmarks on deno with Go and also extended the profiling options in order to make it easier to find potential bottlenecks (#193). The Go profiling tools provide a great overview and you can visualize stuff like this:

I've been working with Go interop for around two years, integrating Python, Lua and other runtimes with a HTTP server (actually an API gateway that people use to handle many requests) and I would say that the overhead of cgo calls is pretty big.
I think that the @ry minimal V8 bindings are pretty good and helps on keeping this overhead as low as possible, at the same time I've been hacking around Rust (and also learning it!) and recently ended up porting this component. I've also started to hack on a minimal deno port that uses std::sync::mpsc (have also tried crossbeam, really exciting stuff) in order to be able to benchmark and potentially compare both implementations. From my Rust experiments it seems that interop is lighter here and this could represent a big performance gain.
You may also find these benchmarks interesting, of course this is experimental and I'm sure there are many ways to optimize this.

[ghost]

@matiasinsaurralde Your PR displays exactly what I feared: Deno will end up clogging up at the JS wrapper performing with horrible throughput. Not even beating Node.js, swapping to Tokio is completely pointless until that wrapper is fixed. The entire Deno project is completely pointless if it's just going to be a worse performing copy of Node.js.

[ry]

@alexhultman Removing Go is to avoid a double GC - which I think everyone can appreciate. And we're experimenting with interfaces - first pass implementations like the HTTP patch let us test. No one is claiming any release or interface commitment.

[matiasinsaurralde]

In case anyone is interested in hacking around, I've pushed a few tweaks for rust-v8worker2 and a limited implementation of Deno code in Rust here (currently you can boot up the runtime and execute a program, only readFileSync is implemented at the moment) 👍

[brandonros]

How do the HTTP benchmarks look when using reno? @matiasinsaurralde

[matiasinsaurralde]

@brandonros I haven't implemented the HTTP module yet, there are many options for this. I've been doing some quick benchmarks with this system call readFileSync (calling this 5000 times).

1. Running the Deno version that uses Go:

% ./deno readfile.ts 
elapsed =  190

2. Running reno using a debug build:

% ./target/debug/reno readfile.ts 
elapsed =  250

3. Running reno using a release build:

% ./target/release/reno readfile.ts 
elapsed =  140

4. NodeJS:

% node readfile_node.js 
elapsed =  77

The main advantage of Rust in this scenario is that we can interact directly with the C data structures, etc.
I've been trying to measure the overhead of using Protocol Buffers as described in #269 and started to migrate our Protocol Buffers definition to Cap'n Proto, implemented the Rust side of it but got stuck trying to parse these messages on the V8 side, found issues with the Cap'n Proto TS library that require some serious debugging.

[brandonros]

@matiasinsaurralde If you had to estimate, do you have faith that switching to Cap'n Proto will bring the elapsed time down from 140, close to Node's 77?

For perspective, it might make sense to show how long the pure C/pure Rust version of the same benchmark would take?

Much like @alexhultman, I'm struggling to see the benefit of running a message-passing layer on top of V8, given the performance. How does node.js currently solve this issue?

[ghost]

For perspective, it might make sense to show how long the pure C/pure Rust version of the same benchmark would take?

For HTTP Deno would currently be about 13x off.

[ssokolow]

Running reno using a releasebuild:

% ./target/release/reno readfile.ts 
elapsed =  140

NodeJS:

% node readfile_node.js 
elapsed =  77

NPM encountered something similar to this during a reliability-oriented rewrite of one of their components and wound up discovering that a surprising amount of the difference was down to how much optimization effort had been put into Node.js in areas you'd underestimate the impact of, like default buffer sizes.

[kentonv]

@matiasinsaurralde Can you tag me on any capnp-ts issues you're running into? I can't make any promises but I'm trying to make some time to play with it myself soon and I hope to help fix issues while I'm at it. (I created Cap'n Proto and the C++ reference implementation but not capnp-ts.)

[ghost]

I just wanted to insert a few points.

It's easy to "dislike" comments that goes against ones gut feeling and fanatism. However disliking someone's comments does not make any change on reality, just because you don't want to accept reality for what it is. You can't pull the plug on reality and the reality as in, what's measurable with scientific methods shows a very clear picture.

If a project stems from the acknowledgement of prior mistakes in Node.js but still clings to it as the only source of reference then you're not going to get any further than a few baby steps past.

Crystal, Golang, Rust, Swift - they all have servers that outperform Node.js by multiple significant times over. You really need to stop comparing with Node.js and raise the stakes if you want to get anywhere real.

I have actual code running inside of reality in this very moment doing about 6x of Node.js, inside of Node.js via V8 native addons. It properly communicates with JavaScript callbacks and methods, has proper URL routing and SSL. It does about 1.5x of Crystal and Golang's fasthttp.

My point is that solely comparing with, and settling for, Node.js is only going to lead you to yet another Node.js. Clearly it is possible to raise performance up to Golang levels from inside V8, just do it already. Comparing with Node.js and accepting even a 4/5th the perf. is simply unacceptable.

Can we please start comparing with Golang's fasthttp or similar?

[matiasinsaurralde]

@kentonv Sure, I will continue my experiments this week and will be able to report the detailed issues on the repo. Thanks.

[brandonros]

@alexhultman

I have actual code running inside of reality in this very moment doing about 6x of Node.js, inside of Node.js via V8 native addons. It properly communicates with JavaScript callbacks and methods, has proper URL routing and SSL. It does about 1.5x of Crystal and Golang's fasthttp.

Link to said code? How does it compare to Rust?

[ghost]

You always pay a quite hefty (but still somewhat acceptable) fine calling JS functions from C++. So of course it is impossible to measure up all the way to a native-only implementation. I lose about 30% req/sec when running inside of Node.js compared to as a stand-alone C++-only build.

Still, 70% of a good C++ implementation is enough to measure up to the fastest available for Golang.

[brandonros]

@alexhultman I want to learn Rust. Let's work together on what you think this project should be. I've tried e-mailing you but you never wrote back.

[ghost]

I don't know a single line of Rust though, so I can't help

[brandonros]

@alexhultman

1. You ignored my comment about the e-mail

2. You ignored my comment about the link to your code that is 1.5x as fast as fasthttp.

3. It sounds like you are saying protocol buffers/Cap 'n Proto are the wrong thing to do... proven by your fast JavaScript code, which uses callbacks and methods and native modules? If you remove node.js out of the picture and take the v8 runtime by itself, how would you compile native modules that hook into the v8 runtime but are still callable by Javascript?

4. I also have full faith you could learn Rust to help

[robbym]

I've been watching this project since Ryan gave the talk, and have been wanting to contribute. I'm fairly familiar with Rust, but not with V8, Cap'n Proto or protobufs (I get the concept though). Is someone taking the helm on the Rust experiment and needs help?

[brandonros]

I'm currently researching what C++ code is required to hook a function into Javascript. So far, I pieced this together from things I found online:

void call_javascript_from_cpp() {
    HandleScope handle_scope;

    Persistent<Context> context = Context::New();
    Context::Scope context_scope(context);
    Handle<String> source = String::New("var testFunction = function() { return arguments[0] + arguments[1]; };");
    Handle<Script> script = Script::Compile(source);
    Handle<Value> result = script->Run();

    Handle<v8::Object> global = context->Global();
    Handle<v8::Value> value = global->Get(String::New("testFunction"));
    Handle<v8::Function> func = v8::Handle<v8::Function>::Cast(value);

    Handle<Value> args[2];
    args[0] = v8::String::New("1");
    args[1] = v8::String::New("1");

    Handle<Value> js_result = func->Call(global, 2, args);

    context.Dispose();
}

void call_cpp_from_javascript() {

}

Once I figure out the second function... I'm guessing we could use https://github.com/alexcrichton/rust-ffi-examples/tree/master/cpp-to-rust

I obviously don't know half as much as other people in this thread, but I was thinking about something simple. Send a message from Javascript to C++, get a response back. Kind of like... pubsub? For... network connections and filesystem operations? I'm sure that'll go great..........

[robbym]

@brandonros Looking at the roadmap, it seems like the portion that will be interacting with the VM, called libdeno, will be written in C++, which will expose a C api so that Rust can bind to it. The beginnings of it are here: https://github.com/ry/deno/blob/master/src/main.rs

As far as the messaging protocol you are mentioning, they already decided to use protobuf (Cap'n Proto in future?) and pub/sub.

[Azareal]

I would be careful about hello world benchmarks, they're notoriously... Useless lol
I suppose, they are alright as a rough indicator, but I wouldn't read into them too much.
In practice, it's largely how well optimised the application is rather than the stack below it.

Go seems to have a fair bit of room for improvement, at-least as far as performance is concerned (there's always some new idea bouncing around their Github), but the GC is probably always going to be a problem lurking round the corner.
The best you can probably get on that front is probably to reuse memory across requests ala fasthttp, I believe one thing fasthttp recommended there were the sync.Pools.

There are a couple of proposals including a percpu sharded value proposal which might also help with performance in some areas, but I'm not sure how much momentum it has behind it.

And there are ideas being bounced around to drastically reduce allocations in Go 2 (e.g. the conversions between []byte and string which can sometimes be costly), although I can't really say much on that, as the details seem a little vague and I'm not sure when it may be released.

All and all though, Rust was practically built for speed, while Go focuses more on productivity, so Go is probably always going to be slower, there is only so much people can do. It really depends on what you're really going for.

[ghost]

I would be careful about hello world benchmarks, they're notoriously... Useless lol
I suppose, they are alright as a rough indicator, but I wouldn't read into them too much.
In practice, it's largely how well optimised the application is rather than the stack below it.

What is a "hello world benchmark" to you? If we're talking about HTTP pipelining of the string "hello world" then I can easily outperform Node.js by 80x (thousands of times over when counting ExpressJS). That's unlikely to ever be a useful number though. The 6x comes from a far more useful case and is not a "hello world" benchmark. Or maybe you suggest we all go write our servers in Ruby on Rails? Then what was the point of Node.js in the first place? What is the point with Deno with that opinion? Just go use Apache/PHP then.

[Azareal]

Edited it as I didn't articulate my thoughts as well as I should have, there are pros and cons to everything. That said, if you're on this side of the VM, then you're probably going for speed more than productivity, otherwise just write the thing in TypeScript lol

As for the performance bit, I was going off:
From his talk: Using a simple hello world benchmark, they achieved almost 2mil req/sec - nearly 250k more than the next best implementation and 500k more than Go's fasthttp. That.

https://www.techempower.com/benchmarks/#section=data-r16&hw=ph&test=json
I've seen benchmarks which put PHP as faster than Rust / C++ before, actually. There's a thing called Swoole which claims to be really fast. It's a very curious thing. Probably thanks to C in that particular case and the narrow thing it benched.

[ghost]

Yeah those 2 mil are complete nonsense. They are achieved using HTTP pipelining which basically only benchmarks the parser/formatter. I get 5 million req/sec in my implementation when I do that.

[ry]

Yes we will likely use Tokio. It's about to land in #434