first implementation of skywire mainnet

CONTRIBUTE.md

@@ -0,0 +1,9 @@
+# How to help:
+
+Interested developers can help by running the software and using the applications to test the functionality. They can file bug
+issues in case something is not working. Bug issues should include a description of the bug, steps to reproduce and should 
+have [Mainnet] in the title so they can be filtered more easily. We will fix bugs as soon as possible. 
+
+As soon as the documentation is released, we will provide guides on how to help the development of Skywire, whether it is a
+new transport implementation or an application developed on Skywire. Bounties will be available for completion of certain
+tasks.

Makefile

@@ -0,0 +1,23 @@
+lint: ## Run linters. Use make install-linters first.
+	GO111MODULE=on vendorcheck ./...
+	GO111MODULE=on golangci-lint run -c .golangci.yml ./...
+	# The govet version in golangci-lint is out of date and has spurious warnings, run it separately
+	GO111MODULE=on go vet -all ./...
+
+install-linters: ## Install linters
+	go get -u github.com/FiloSottile/vendorcheck
+	# For some reason this install method is not recommended, see https://github.com/golangci/golangci-lint#install
+	# However, they suggest `curl ... | bash` which we should not do
+	go get -u github.com/golangci/golangci-lint/cmd/golangci-lint
+
+format: ## Formats the code. Must have goimports installed (use make install-linters).
+	goimports -w -local github.com/skycoin/skywire ./pkg
+	goimports -w -local github.com/skycoin/skywire ./cmd
+	goimports -w -local github.com/skycoin/skywire ./internal
+
+dep: ## sorts dependencies
+	GO111MODULE=on go mod vendor -v
+
+test: ## Run tests for net
+	GO111MODULE=on go test -race -tags no_ci -cover -timeout=5m ./internal/...
+	GO111MODULE=on go test -race -tags no_ci -cover -timeout=5m ./pkg/...

README.md

@@ -0,0 +1,145 @@
+[![Build Status](https://travis-ci.com/watercompany/skywire.svg?token=QxVQj6gVZDzoFxD2YG65&branch=master)](https://travis-ci.com/watercompany/skywire)
+
+# Skywire Mainnet - Public Test Phase
+
+## Notes on this release
+
+This is a public testing version of the Skywire mainnet and is intended for developers use to find bugs only. It is not yet intended to replace the testnet and miners should not install this software on their miners or they may lose their reward eligibility. 
+
+The software is still under heavy development and the current version is intended for public testing purposes only. A GUI interface and various guides on how to use Skywire, application development on Skywire and contribution policies will follow in the near future. For now this version of the software can be used by developers to test the functionality and file bug issues to help the development. 
+
+## Architecture 
+
+Skywire is a decentralized and private network. Skywire separates the data and control plane of the network and assigns the tasks of network coordination and administration to dedicated services, while the nodes follow the rules that were created by the control plane and execute them. 
+
+The core of Skywire is the Skywire node which hosts applications and is the gateway to use the network. It establishes connections, called transports, to other nodes, requests the setup of routes and forwards packets for other nodes on a route. The Skywire node exposes an API to applications for using the networking protocol of Skywire. 
+
+In order to detach control plane tasks from the network nodes, there are 3 other services that maintain a picture of the network topology, calculate routes (currently based on the number of hops, but will be extended to other metrics) and set the routing rules on the nodes. 
+
+The transport discovery maintains a picture of the network topology, by allowing Skywire nodes to advertise transports that they established with other nodes. It also allows to upload a status to indicate whether a given transport is currently working or not.
+
+On the basis of this information the route finder calculates the most efficient route in the network. Nodes request a route to a given public key and the route finder will calculate the best route and return the transports that the packet will be sent over to reach the intended node. 
+
+This information is sent from a node to the Setup Node, which sets the routing rules in all nodes along a route. Skywire nodes determine, which nodes they accept routing rules from, so only a whitelisted node can send routing rules to a node in the network. The only information the Skywire node gets for routing is a Routing ID and an associated rule that defines which transport to send a packet to (or to consume the packet). Therefore nodes along a route only know the last and next hop along the route, but not where the packet originates from and where it is sent to. Skywire supports source routing, so nodes can specify a path that a packet is supposed to take in the network. 
+
+There are currently two types of transports that nodes can use. The messaging transport is a transport between two nodes that uses an intermediary messaging server to relay packets between them. The connection to a specific node and the connection to a messaging server is facilitated by a discovery service, that allows nodes to advertise the messaging servers over which they can be contacted. This transport is used by the setup node to send routing rules and can be used for other applications as well. It allows nodes behind NATs to communicate. The second transport type is TCP, which sets up a connection between two servers with a public IP. More transport types will be supported in the future and custom transport implementations can be written for specific use cases.
+
+## Build and run
+
+### Requirements
+
+Skywire requires a version of [golang](https://golang.org/) with [go modules](https://github.com/golang/go/wiki/Modules) support.
+
+### Build
+
+```bash
+# Clone.
+$ git clone https://github.com/skycoin/skywire
+$ cd skywire
+$ git checkout mainnet-develop
+
+# Install skywire-node, skywire-cli, manager-node and therealssh-cli
+$ GO111MODULE=on go install ./cmd/skywire-node ./cmd/skywire-cli ./cmd/manager-node ./cmd/therealssh-cli
+
+# Setup run folder.
+mkdir skywire
+mkdir apps
+
+# Build apps.
+$ GO111MODULE=on go build -o ./apps/chat.v1.0 ./cmd/apps/chat
+$ GO111MODULE=on go build -o ./apps/helloworld.v1.0 ./cmd/apps/helloworld
+$ GO111MODULE=on go build -o ./apps/therealproxy.v1.0 ./cmd/apps/therealproxy
+$ GO111MODULE=on go build -o ./apps/therealproxy-client.v1.0  ./cmd/apps/therealproxy-client
+$ GO111MODULE=on go build -o ./apps/therealssh.v1.0  ./cmd/apps/therealssh
+$ GO111MODULE=on go build -o ./apps/therealssh-client.v1.0  ./cmd/apps/therealssh-client
+
+# Generate default json config.
+$ skywire-cli config
+```
+
+### Run `skywire-node`
+
+`skywire-node` hosts apps, proxies app's requests to remote nodes and exposes communication API that apps can use to implement communication protocols. App binaries are spawned by the node, communication between node and app is performed via unix pipes provided on app startup.
+
+```bash
+# Run skywire-node. It takes one argument; the path of a configuration file (`skywire.json` if unspecified).
+$ skywire-node skywire.json
+```
+
+### Run `skywire-cli`
+
+The `skywire-cli` tool is used to control the `skywire-node`. Refer to the help menu for usage:
+
+```bash
+$ skywire-cli -h
+
+# Command Line Interface for skywire
+# 
+# Usage:
+#   skywire-cli [command]
+# 
+# Available Commands:
+#   app                 app management operations
+#   config              Generate default config file
+#   help                Help about any command
+#   messaging-discovery manage operations with messaging discovery api
+#   route-finder        manage operations with route finder api
+#   routing-rules       manages operations with routing rules
+#   transport-discovery manage operations with transport discovery api
+#   transports          manages transports related operations
+# 
+# Flags:
+#   -h, --help         help for skywire-cli
+#       --rpc string   RPC server address (default "localhost:3435")
+# 
+# Use "skywire-cli [command] --help" for more information about a command.
+```
+
+### Apps
+
+After `skywire-node` is up and running with default environment, default apps are run with the configuration specified in `skywire.json`. Refer to the following for usage of the default apps:
+
+- [Chat](/cmd/apps/chat)
+- [Hello World](/cmd/apps/helloworld)
+- [The Real Proxy](/cmd/apps/therealproxy) ([Client](/cmd/apps/therealproxy-client))
+- [The Real SSH](/cmd/apps/therealssh) ([Client](/cmd/apps/therealssh-client))
+
+### Transports
+
+In order for a local Skywire App to communicate with an App running on a remote Skywire node, a transport to that remote Skywire node needs to be established.
+
+Transports can be established via the `skywire-cli`.
+
+```bash
+# Establish transport to `0276ad1c5e77d7945ad6343a3c36a8014f463653b3375b6e02ebeaa3a21d89e881`.
+$ skywire-cli transports add 0276ad1c5e77d7945ad6343a3c36a8014f463653b3375b6e02ebeaa3a21d89e881
+
+# List established transports.
+$ skywire-cli transports list
+```
+
+## App programming API
+
+App is a generic binary that can be executed by the node. On app
+startup node will open pair of unix pipes that will be used for
+communication between app and node. `app` packages exposes
+communication API over the pipe.
+
+```golang
+// Config defines configuration parameters for App
+&app.Config{AppName: "helloworld", AppVersion: "1.0", ProtocolVersion: "0.0.1"}
+// Setup setups app using default pair of pipes
+func Setup(config *Config) (*App, error) {}
+
+// Accept awaits for incoming loop confirmation request from a Node and
+// returns net.Conn for a received loop.
+func (app *App) Accept() (net.Conn, error) {}
+
+// Addr implements net.Addr for App connections.
+&Addr{PubKey: pk, Port: 12}
+// Dial sends create loop request to a Node and returns net.Conn for created loop.
+func (app *App) Dial(raddr *Addr) (net.Conn, error) {}
+
+// Close implements io.Closer for App.
+func (app *App) Close() error {}
+```

ROADMAP.md

@@ -0,0 +1,12 @@
+# Next steps:
+
+Detailed documentation of the software will follow soon and will be updated as the development continues. The next development steps are: 
+
+- [ ] implement bandwidth settlement for monetization of network services
+- [ ] CHB / CoinJoin 
+- [ ] new UI for the manager
+- [ ] Remote manager that allows management of unlimited nodes from anywhere
+- [ ] Run CXO on Skywire
+- [ ] Create service discovery for peer discovery
+- [ ] Documentation and guides on development on Skywire, application development, how to use it and the architecture itself
+- [ ] Integrate Skywire with package manager

cmd/apps/chat/README.org

@@ -0,0 +1,44 @@
+* Skywire Chat app
+
+Chat implements basic text messaging between skywire nodes. Messaging
+UI is exposed via web interface. Chat only supports one WEB client
+user at a time.
+
+** Local setup
+
+Create 2 node config files:
+
+~skywire1.json~
+#+BEGIN_SRC js
+  "apps": [
+    {
+      "app": "chat",
+      "version": "1.0",
+      "auto_start": true,
+      "port": 1
+    }
+  ]
+#+END_SRC
+
+~skywire2.json~
+#+BEGIN_SRC js
+  "apps": [
+    {
+      "app": "chat",
+      "version": "1.0",
+      "auto_start": true,
+      "port": 1,
+      "args": ["-addr", ":8001"]
+    }
+  ]
+#+END_SRC
+
+Compile binaries and start 2 nodes:
+
+#+BEGIN_SRC bash
+$ go build -o apps/chat.v1.0 ./cmd/apps/chat
+$ ./skywire-node skywire1.json
+$ ./skywire-node skywire2.json
+#+END_SRC
+
+Chat interface will be available on ports ~8000~ and ~8001~.

cmd/apps/chat/chat.go

@@ -0,0 +1,150 @@
+//go:generate esc -o static.go -prefix static static
+
+/*
+chat app for skywire node
+*/
+package main
+
+import (
+	"encoding/json"
+	"flag"
+	"fmt"
+	"log"
+	"net"
+	"net/http"
+	"sync"
+
+	"github.com/skycoin/skywire/pkg/app"
+	"github.com/skycoin/skywire/pkg/cipher"
+)
+
+var addr = flag.String("addr", ":8000", "address to bind")
+
+var (
+	chatApp    *app.App
+	clientChan chan string
+	chatConns  map[cipher.PubKey]net.Conn
+	connsMu    sync.Mutex
+)
+
+func main() {
+	flag.Parse()
+
+	var err error
+	config := &app.Config{AppName: "chat", AppVersion: "1.0", ProtocolVersion: "0.0.1"}
+	chatApp, err = app.Setup(config)
+	if err != nil {
+		log.Fatal("Setup failure: ", err)
+	}
+	defer chatApp.Close()
+
+	chatConns = make(map[cipher.PubKey]net.Conn)
+	go listenLoop()
+
+	http.Handle("/", http.FileServer(FS(false)))
+	http.HandleFunc("/message", messageHandler)
+	http.HandleFunc("/sse", sseHandler)
+
+	log.Println("Serving HTTP on", *addr)
+	log.Fatal(http.ListenAndServe(*addr, nil))
+}
+
+func listenLoop() {
+	for {
+		conn, err := chatApp.Accept()
+		if err != nil {
+			log.Println("failed to accept conn: ", err)
+			return
+		}
+
+		raddr := conn.RemoteAddr().(*app.Addr)
+		connsMu.Lock()
+		chatConns[raddr.PubKey] = conn
+		connsMu.Unlock()
+
+		go handleConn(conn)
+	}
+}
+
+func handleConn(conn net.Conn) {
+	raddr := conn.RemoteAddr().(*app.Addr)
+	for {
+		buf := make([]byte, 32*1024)
+		n, err := conn.Read(buf)
+		if err != nil {
+			log.Println("failed to read packet: ", err)
+			return
+		}
+
+		clientMsg, _ := json.Marshal(map[string]string{"sender": raddr.PubKey.Hex(), "message": string(buf[:n])}) // nolint
+		select {
+		case clientChan <- string(clientMsg):
+		default:
+		}
+	}
+}
+
+func messageHandler(w http.ResponseWriter, req *http.Request) {
+	data := map[string]string{}
+	if err := json.NewDecoder(req.Body).Decode(&data); err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	pk := cipher.PubKey{}
+	if err := pk.UnmarshalText([]byte(data["recipient"])); err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	addr := &app.Addr{PubKey: pk, Port: 1}
+	connsMu.Lock()
+	conn := chatConns[pk]
+	connsMu.Unlock()
+
+	if conn == nil {
+		var err error
+		conn, err = chatApp.Dial(addr)
+		if err != nil {
+			http.Error(w, err.Error(), http.StatusBadRequest)
+			return
+		}
+
+		connsMu.Lock()
+		chatConns[pk] = conn
+		connsMu.Unlock()
+
+		go handleConn(conn)
+	}
+
+	if _, err := conn.Write([]byte(data["message"])); err != nil {
+		http.Error(w, err.Error(), http.StatusBadRequest)
+		return
+	}
+}
+
+func sseHandler(w http.ResponseWriter, req *http.Request) {
+	f, ok := w.(http.Flusher)
+	if !ok {
+		http.Error(w, "Streaming unsupported!", http.StatusBadRequest)
+		return
+	}
+
+	clientChan = make(chan string)
+	go func() {
+		<-req.Context().Done()
+		close(clientChan)
+		clientChan = nil
+		log.Println("SSE connection were closed.")
+	}()
+
+	w.Header().Set("Content-Type", "text/event-stream")
+	w.Header().Set("Cache-Control", "no-cache")
+	w.Header().Set("Connection", "keep-alive")
+	w.Header().Set("Transfer-Encoding", "chunked")
+
+	for msg := range clientChan {
+		fmt.Fprintf(w, "data: %s\n\n", msg)
+		f.Flush()
+	}
+}