Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate m3 dsmgpty with hypervisor and various fixes. #166

Merged
merged 33 commits into from
Mar 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
8cda7a1
Refactored dmsgpty for easier readability.
Jan 30, 2020
874bb73
Merge branch 'milestone2' of github.com:SkycoinProject/skywire-mainne…
Feb 21, 2020
f896ef1
Changed to use new dmsgpty.
Feb 21, 2020
a03f7af
Vendor
Feb 21, 2020
1699469
Tidy vendor.
Feb 21, 2020
04ade12
Paused progress on hypervisor endpoints refactor.
Feb 24, 2020
68c8de2
Re-implemented RPCClientDialer.
Feb 25, 2020
69dab69
Integrated dmsgpty into hypervisor.
Feb 25, 2020
ec482bc
Format.
Feb 25, 2020
e44d75d
Remove legacy integration folder and update vendor.
Feb 25, 2020
c63edc1
Changes to integrate dmsgpty
Feb 26, 2020
487296b
First working implementation of dmsgpty from hypervisor.
Feb 27, 2020
18f482d
Tidy vendor.
Feb 27, 2020
09c5eb7
Make hypervisor getVisors concurrent.
Feb 28, 2020
299f80f
Tidy vendor.
Feb 28, 2020
bc6f38e
Made changes as requested by @nkryuchkov
Feb 28, 2020
155be22
Fix bug where managedTransport may not close properly (#160).
Feb 28, 2020
36fa94c
Fix conflicts.
Mar 2, 2020
bdc72de
Made hypervisor ping endpoint a separate function.
Mar 2, 2020
40b7792
Further changes to fix #160
Mar 2, 2020
c769c60
Fix merge conflicts.
Mar 2, 2020
226086d
Update vendor.
Mar 2, 2020
f92427b
Fix logging.
Mar 2, 2020
e847cae
Improved logging.
Mar 2, 2020
1830d40
Improve logging.
Mar 2, 2020
447862e
Added exponential backoff to mTp.redial and various fixes.
Mar 3, 2020
0e696f0
Format.
Mar 3, 2020
b16defe
Make travis happy.
Mar 3, 2020
bf93616
Changes as suggested by @nkryuchkov and @Darkren
Mar 3, 2020
33ec06d
Fix merge conflicts.
Mar 4, 2020
608c380
Separate /cmd/hypervisor/commands/root.go:rootCmd() into smaller funcs.
Mar 4, 2020
047271b
Update vendor.
Mar 4, 2020
8341370
Update vendor.
Mar 4, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,6 @@ bin: ## Build `skywire-visor`, `skywire-cli`, `hypervisor`
${OPTS} go build ${BUILD_OPTS} -o ./skywire-cli ./cmd/skywire-cli
${OPTS} go build ${BUILD_OPTS} -o ./setup-node ./cmd/setup-node
${OPTS} go build ${BUILD_OPTS} -o ./hypervisor ./cmd/hypervisor
${OPTS} go build ${BUILD_OPTS} -o ./dmsgpty ./cmd/dmsgpty

release: ## Build `skywire-visor`, `skywire-cli`, `hypervisor` and apps without -race flag
${OPTS} go build ${BUILD_OPTS} -o ./skywire-visor ./cmd/skywire-visor
Expand Down Expand Up @@ -186,21 +185,6 @@ run-syslog: ## Run syslog-ng in docker. Logs are mounted under /tmp/syslog
-docker container rm syslog-ng -f
docker run -d -p 514:514/udp -v /tmp/syslog:/var/log --name syslog-ng balabit/syslog-ng:latest

integration-startup: ## Starts up the required transports between `skywire-visor`s of interactive testing environment
./integration/startup.sh

integration-teardown: ## Tears down all saved configs and states of integration executables
./integration/tear-down.sh

integration-run-generic: ## Runs the generic interactive testing environment
./integration/run-generic-env.sh

integration-run-messaging: ## Runs the messaging interactive testing environment
./integration/run-messaging-env.sh

integration-run-proxy: ## Runs the proxy interactive testing environment
./integration/run-proxy-env.sh

mod-comm: ## Comments the 'replace' rule in go.mod
./ci_scripts/go_mod_replace.sh comment go.mod

Expand Down
88 changes: 0 additions & 88 deletions cmd/dmsgpty/commands/root.go

This file was deleted.

99 changes: 0 additions & 99 deletions cmd/dmsgpty/commands/whitelist.go

This file was deleted.

9 changes: 0 additions & 9 deletions cmd/dmsgpty/dmsgpty.go

This file was deleted.

106 changes: 54 additions & 52 deletions cmd/hypervisor/commands/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (

"github.com/SkycoinProject/dmsg"
"github.com/SkycoinProject/dmsg/disc"

"github.com/SkycoinProject/skycoin/src/util/logging"
"github.com/spf13/cobra"

Expand Down Expand Up @@ -49,67 +48,70 @@ var rootCmd = &cobra.Command{
log.Printf("Failed to output build info: %v", err)
}

if configPath == "" {
configPath = pathutil.FindConfigPath(args, -1, configEnv, pathutil.HypervisorDefaults())
}

var config hypervisor.Config
config.FillDefaults()
if err := config.Parse(configPath); err != nil {
log.WithError(err).Fatalln("failed to parse config file")
}

fmt.Println("Config: \n", config)

var (
httpAddr = config.Interfaces.HTTPAddr
rpcAddr = config.Interfaces.RPCAddr
)
conf := prepareConfig(args)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function looks nice! 🙂


m, err := hypervisor.New(config)
// Prepare hypervisor.
hv, err := hypervisor.New(conf)
if err != nil {
log.Fatalln("Failed to start hypervisor:", err)
}
if mock {
prepareMockData(hv)
} else {
prepareDmsg(hv, conf)
}

log.Infof("serving RPC on '%s'", rpcAddr)
go func() {
_, rpcPort, err := config.Interfaces.SplitRPCAddr()
if err != nil {
log.Fatalln("Failed to parse rpc port from rpc address:", err)
}

dmsgC := dmsg.NewClient(config.PK, config.SK, disc.NewHTTP(config.DmsgDiscovery), dmsg.DefaultConfig())
go dmsgC.Serve()
// Serve HTTP.
log.WithField("http_addr", conf.HTTPAddr).Info("Serving HTTP.")
if err := http.ListenAndServe(conf.HTTPAddr, hv); err != nil {
log.WithError(err).Fatal("Hypervisor exited with error.")
}
log.Info("Good bye!")
},
}

l, err := dmsgC.Listen(rpcPort)
if err != nil {
log.Fatalln("Failed to bind tcp port:", err)
}
func prepareConfig(args []string) (conf hypervisor.Config) {
if configPath == "" {
configPath = pathutil.FindConfigPath(args, -1, configEnv, pathutil.HypervisorDefaults())
}
conf.FillDefaults(mock)
if err := conf.Parse(configPath); err != nil {
log.WithError(err).Fatalln("failed to parse config file")
}
log.WithField("config", conf).Info()
return conf
}

if err := m.ServeRPC(l); err != nil {
log.Fatalln("Failed to serve RPC:", err)
}
}()
func prepareMockData(hv *hypervisor.Hypervisor) {
err := hv.AddMockData(hypervisor.MockConfig{
Visors: mockVisors,
MaxTpsPerVisor: mockMaxTps,
MaxRoutesPerVisor: mockMaxRoutes,
EnableAuth: mockEnableAuth,
})
if err != nil {
log.Fatalln("Failed to add mock data:", err)
}
}

if mock {
err := m.AddMockData(hypervisor.MockConfig{
Visors: mockVisors,
MaxTpsPerVisor: mockMaxTps,
MaxRoutesPerVisor: mockMaxRoutes,
EnableAuth: mockEnableAuth,
})
if err != nil {
log.Fatalln("Failed to add mock data:", err)
}
}
func prepareDmsg(hv *hypervisor.Hypervisor, conf hypervisor.Config) {
// Prepare dmsg client.
dmsgC := dmsg.NewClient(conf.PK, conf.SK, disc.NewHTTP(conf.DmsgDiscovery), dmsg.DefaultConfig())
go dmsgC.Serve()

log.Infof("serving HTTP on '%s'", httpAddr)
if err := http.ListenAndServe(httpAddr, m); err != nil {
log.Fatalln("Hypervisor exited with error:", err)
dmsgL, err := dmsgC.Listen(conf.DmsgPort)
if err != nil {
log.WithField("addr", fmt.Sprintf("dmsg://%s:%d", conf.PK, conf.DmsgPort)).
Fatal("Failed to listen over dmsg.")
}
go func() {
if err := hv.ServeRPC(dmsgC, dmsgL); err != nil {
log.WithError(err).
Fatal("Failed to serve RPC client over dmsg.")
}

log.Println("Good bye!")
},
}()
log.WithField("addr", fmt.Sprintf("dmsg://%s:%d", conf.PK, conf.DmsgPort)).
Info("Serving RPC client over dmsg.")
}

// Execute executes root CLI command.
Expand Down
2 changes: 1 addition & 1 deletion cmd/skywire-cli/commands/visor/gen-config.go
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ func defaultConfig() *visor.Config {

func defaultDmsgPtyConfig() visor.DmsgPtyConfig {
return visor.DmsgPtyConfig{
Port: skyenv.DefaultDmsgPtyPort,
Port: skyenv.DmsgPtyPort,
AuthFile: "./skywire/dmsgpty/whitelist.json",
CLINet: skyenv.DefaultDmsgPtyCLINet,
CLIAddr: skyenv.DefaultDmsgPtyCLIAddr,
Expand Down
6 changes: 2 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,12 @@ module github.com/SkycoinProject/skywire-mainnet
go 1.13

require (
github.com/SkycoinProject/dmsg v0.0.0-20200303104641-cfc70993f6b0
github.com/SkycoinProject/dmsg v0.0.0-20200304081751-cf4d40f5428f
github.com/SkycoinProject/skycoin v0.27.0
github.com/SkycoinProject/yamux v0.0.0-20191213015001-a36efeefbf6a
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5
github.com/creack/pty v1.1.9
github.com/go-chi/chi v4.0.2+incompatible
github.com/google/uuid v1.1.1
github.com/gorilla/handlers v1.4.2
github.com/gorilla/securecookie v1.1.1
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/kr/pretty v0.2.0 // indirect
Expand All @@ -24,7 +22,7 @@ require (
github.com/spf13/cobra v0.0.5
github.com/stretchr/testify v1.4.0
go.etcd.io/bbolt v1.3.3
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073 // indirect
golang.org/x/net v0.0.0-20191204025024-5ee1b9f4859a
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527 // indirect
)
Expand Down
Loading