Establish an operational structure for the WG

[aturon]

Given how large the WG is, and the number of topics covered, it seems best to try to carve out some relatively clear subgroups, with a clear lead/point person and scope.

This thread is for discussing what such a structure might look like.

[aturon]

There are a couple of subgroups that I think are relatively clear-cut:

• "Core async": covering futures, async/await notation, Tokio (and other event loops)
• "Low-level" networking: work at the level of implementing TCP (and similar); see Low-level networking #15
• "Web apps": making sense of the space of web frameworks in Rust, contributing back to them and ensuring that we have a solid offering for Rust 2018

Then there are a bunch of additional topics that it's hard to know how best to "slice":

• "Protocol support", ranging from http, grpc to redis, s3 etc. It seems too big to have this be a single group, but not clear how best to slice. Server vs client-side?

• "Middleware", as exemplified by the Tower project. Not limited to http; there's a lot of generic networking components we could offer at this level.

• "Guidelines": we clearly need to provide ecosystem guidelines for the above topics, but it's not clear whether that should be factored out as a separate subgroup, or treated as part of the work each group is doing in concert.

Thoughts on how to divide these, or whether important areas are missing?

[yoshuawuyts]

I feel there might be a section missing for things that are related to networking, but might not necessarily fit any of the above buckets.

For example one of my current side projects is porting Erlang's supervisor library to Rust to improve the error handling story for web apps. I'm not sure if this would fit in under "Low-level" or "Web Apps". It could be either. It could be neither. But I feel there would be value in finding the right place to collaborate with others on things like these!

Other Important Topics

I don't know if the following topics strictly fall within the Net WG's scope, but I do think they're crucial for a good web app story:

• Logging: How can we ensure that no warning goes unnoticed in production? Do we have integrations with the most common log aggregators? What is the compatibility between the log and slog crates? Are there other things we should be aware of?
• Testing: How can we ensure that testing web apps is delightful in Rust? network playback (alt), network emulation, and other testing strategies are crucial to create reliable applications.
• Error handling: people probably initially think of failure, but retrying errors, and supervisor strategies matter a lot for real-world applications.
• Documentation: how can we document everything a web app does? There's common formats for this (e.g. OpenAPI), but how far is the tooling around that? Should any of this integrate with RustDoc / Doxidize?

Finding the right division

Then there are a bunch of additional topics that it's hard to know how best to "slice" (..)

I agree with this: it's hard to know up front what the right way to group people is. My personal fear is that if we start off with a high amount of subgroups it becomes both harder to get involved, and it might become tricky to figure out where to contribute.

For example I think "Middleware" and "Web Apps" are very closely related. I feel "find a common solution for middleware" could very well be a concrete goal for the Web Apps WG!

Conclusion

I'm having a bit of trouble figuring out the answers to the questions. It might be because it's not entirely clear to me what the "North Star" of the WG is. What are we trying to achieve? What do we consider the definition of "Network"? What is within the scope of the WG, and what isn't? And in turn, what should we prioritize for the 2018 release?

Maybe it would be useful to write a bit more about what we want to achieve with the WG before we create subgroups?

[aturon]

@yoshuawuyts

My personal fear is that if we start off with a high amount of subgroups it becomes both harder to get involved, and it might become tricky to figure out where to contribute.

Yes indeed. That said it feels pretty clear to me that we're going to want some amount of subdivision due to the large number of people (almost 40) and the clear breadth of topics.

Maybe it would be useful to write a bit more about what we want to achieve with the WG before we create subgroups?

Totally agreed! I wanted to have both threads proceeding in parallel in case folks had insights about structuring, but nailing down the product vision is the most important step.

[bIgBV]

I don't have a lot to say about the division of the group except that at a high level, I like what @aturon proposes. I feel like protocol work can slot in between "low-level" and "Web apps" basically bridging the gap.

One thing I have some thoughts on is the general approach to introducing new programmers to networking in rust. Let me know if this isn't the right forum for this, I'll create a new issue/post on a different one if required. Sync or async. Right now, as far as I can tell there isn't one place a programmer new to rust can go to get a high-level view of the ecosystem. There is great documentation in getting started with rust, and individual projects have great documentation, but unless you know of them already, you wouldn't know how to find them.

Proposal

• I propose that we have a website for rust networking/a page on the official language website. * The topics should be split up based on which layer of the network stack they pertain to.
• The information presented by these topics could be as such:
  • Information regarding getting started with building something at that layer
  • Point out the current best practices and point to the libraries recommended by the community
  • Point to further resources that someone can use when working at that layer

I feel like this approach will cater to users at all levels of the networking story. When the website is first built, the topics needn't have extensive examples and documentation but just point to the right resources. This would mean that someone who wants to build a web application in rust wouldn't have to search for Reddit threads titled "what library do you use to build web applications", which I am guessing a lot of people currently do, and I myself have done at one point.

I used web applications just as an example, the same idea could be applied to people writing protocols at the application layer or even lower when writing custom protocols on top of IP.

Conclusion

Having a central website for everyone to start their networking journey at would be great at helping programmers find the information they are looking for and also help the community consolidate all the information regarding networking in rust.

[Yarn]

@bIgBV i made an issue and shamelessly reposted your post there
#37

[andrewtj]

Perhaps it'd help to flesh out what work is needed in order to clarify what the additional topics should be. A one question survey run for a short period (72 hours) might be a good way to get some input. (I'd be happy to help with summarising feedback.)

For myself, I've been working on a DNS library and having squared off the non-network side of things am now starting to play with Futures and Tokio. The first gap I've found is that there's no first party support for UDP listeners bound to 0.0.0.0/:: (where you need to obtain the destination at receive time and set it when responding). I don't know if this is a one-off that's best resolved with a couple of PRs or if there's more work to do in the UDP area.

I also wonder if there's room for a general name (and possibly service) resolution abstraction. If there is a need (and I'm not sure that there is), that would need to be baked into the ecosystem early on if it's going to have a chance at being useful.

[jsgf]

I had been thinking that "debugging and instrumentation" would be part of the core async subgroup, but since its so cross-cutting I wonder if it should be its own subgroup? I expect that at least some of the implementation would be in the core, but perhaps not all (ie, hooks in the core, other crates to implement higher-level functionality).

[aturon]

Thanks everybody for the discussion! I've now updated the README to reflect the structure and goals we settled on last week, together with elaborated summaries and vision statements from each subgroup.

Closing this out as resolved!