Skip to content

Commit

Permalink
To create profiling data add flag --pprof/-p for skywire-node.
Browse files Browse the repository at this point in the history
Possible values:
- cpu
- mem
- mutex
- block

pprof-files will be generated on exit in ./logs/[--tag]/[mode].pprof

To generate a report use `go tool pprof`

E.g.

```bash
$ ./bin/skywire-node ./integration/generic/nodeA.json --tag NodeA --pprof
cpu
$ go tool pprof --pdf ~/bin/skywire-node ./logs/nodeA/cpu.pprof
```

Run skywire-node with `--pprof http --pport 6060`
(--pport has default value 6060 and could be ommitted when only one node
profiled)

E.g.
```bash
$ skywire-node ./integration/generic/nodeA.json --tag NodeA --pprof http
$ go tool http://localhost:6060/debug/pprof/
```

Read:

- https://golang.org/pkg/net/http/pprof/
- https://blog.golang.org/profiling-go-programs
- https://godoc.org/github.com/pkg/profile

Motivation:

Need to switch on/off `-race` flag for various tests and various profiles.

Now it's possible:

```bash
$ export BUILD_OPTS=
$ make build  # or `make integration-build`
```

2. Environment variable $TEST_OPTS with default value: "-race -tags no_ci -cover -timeout=5m"

Motivation:

- `-timeout=5m` - is a good, safe timeout  for travis-ci
- for developer environments it's more comfortable to use 60s (maximum
90s)

Now it's possible:

```bash
$ export TEST_OPTS="-race -tags no_ci -timeout=60s"                                                                                                  5:41
$ make test
```
  • Loading branch information
ayuryshev committed Jun 9, 2019
1 parent bcae8da commit fe56a51
Show file tree
Hide file tree
Showing 15 changed files with 455 additions and 26 deletions.
50 changes: 25 additions & 25 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ DOCKER_IMAGE?=skywire-runner # docker image to use for running skywire-node.`gol
DOCKER_NETWORK?=SKYNET
DOCKER_NODE?=SKY01
DOCKER_OPTS?=GO111MODULE=on GOOS=linux # go options for compiling for docker container
TEST_OPTS?=-race -tags no_ci -cover -timeout=5m
BUILD_OPTS?=-race

check: lint test ## Run linters and tests

Expand Down Expand Up @@ -59,19 +61,19 @@ vendorcheck: ## Run vendorcheck

test: ## Run tests
-go clean -testcache &>/dev/null
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./internal/...
${OPTS} go test ${TEST_OPTS} ./internal/...
#${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/app/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/cipher/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/manager/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/messaging-discovery/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/node/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/route-finder/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/router/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/routing/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/setup/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/transport/...
${OPTS} go test -race -tags no_ci -cover -timeout=5m ./pkg/transport-discovery/...
${OPTS} go test ${TEST_OPTS} ./pkg/app/...
${OPTS} go test ${TEST_OPTS} ./pkg/cipher/...
${OPTS} go test ${TEST_OPTS} ./pkg/manager/...
${OPTS} go test ${TEST_OPTS} ./pkg/messaging-discovery/...
${OPTS} go test ${TEST_OPTS} ./pkg/node/...
${OPTS} go test ${TEST_OPTS} ./pkg/route-finder/...
${OPTS} go test ${TEST_OPTS} ./pkg/router/...
${OPTS} go test ${TEST_OPTS} ./pkg/routing/...
${OPTS} go test ${TEST_OPTS} ./pkg/setup/...
${OPTS} go test ${TEST_OPTS} ./pkg/transport/...
${OPTS} go test ${TEST_OPTS} ./pkg/transport-discovery/...
${OPTS} go test -tags no_ci -cover -timeout=5m ./pkg/messaging/...


Expand All @@ -93,20 +95,20 @@ dep: ## Sorts dependencies

# Apps
host-apps: ## Build app
${OPTS} go build -race -o ./apps/skychat.v1.0 ./cmd/apps/skychat
${OPTS} go build -race -o ./apps/helloworld.v1.0 ./cmd/apps/helloworld
${OPTS} go build -race -o ./apps/socksproxy.v1.0 ./cmd/apps/therealproxy
${OPTS} go build -race -o ./apps/socksproxy-client.v1.0 ./cmd/apps/therealproxy-client
${OPTS} go build -race -o ./apps/SSH.v1.0 ./cmd/apps/therealssh
${OPTS} go build -race -o ./apps/SSH-client.v1.0 ./cmd/apps/therealssh-client
${OPTS} go build ${BUILD_OPTS} -o ./apps/skychat.v1.0 ./cmd/apps/skychat
${OPTS} go build ${BUILD_OPTS} -o ./apps/helloworld.v1.0 ./cmd/apps/helloworld
${OPTS} go build ${BUILD_OPTS} -o ./apps/socksproxy.v1.0 ./cmd/apps/therealproxy
${OPTS} go build ${BUILD_OPTS} -o ./apps/socksproxy-client.v1.0 ./cmd/apps/therealproxy-client
${OPTS} go build ${BUILD_OPTS} -o ./apps/SSH.v1.0 ./cmd/apps/therealssh
${OPTS} go build ${BUILD_OPTS} -o ./apps/SSH-client.v1.0 ./cmd/apps/therealssh-client

# Bin
bin: ## Build `skywire-node`, `skywire-cli`, `manager-node`, `SSH-cli`
${OPTS} go build -race -o ./skywire-node ./cmd/skywire-node
${OPTS} go build -race -o ./skywire-cli ./cmd/skywire-cli
${OPTS} go build -race -o ./setup-node ./cmd/setup-node
${OPTS} go build -race -o ./manager-node ./cmd/manager-node
${OPTS} go build -race -o ./SSH-cli ./cmd/therealssh-cli
${OPTS} go build ${BUILD_OPTS} -o ./skywire-node ./cmd/skywire-node
${OPTS} go build ${BUILD_OPTS} -o ./skywire-cli ./cmd/skywire-cli
${OPTS} go build ${BUILD_OPTS} -o ./setup-node ./cmd/setup-node
${OPTS} go build ${BUILD_OPTS} -o ./manager-node ./cmd/manager-node
${OPTS} go build ${BUILD_OPTS} -o ./SSH-cli ./cmd/therealssh-cli

release: ## Build skywire-node`, skywire-cli, manager-node, SSH-cli and apps without -race flag
${OPTS} go build -o ./skywire-node ./cmd/skywire-node
Expand All @@ -121,8 +123,6 @@ release: ## Build skywire-node`, skywire-cli, manager-node, SSH-cli and apps wit
${OPTS} go build -o ./apps/SSH.v1.0 ./cmd/apps/therealssh
${OPTS} go build -o ./apps/SSH-client.v1.0 ./cmd/apps/therealssh-client



# Dockerized skywire-node
docker-image: ## Build docker image `skywire-runner`
docker image build --tag=skywire-runner --rm - < skywire-runner.Dockerfile
Expand Down
29 changes: 29 additions & 0 deletions cmd/skywire-node/commands/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package commands
import (
"bufio"
"encoding/json"
"fmt"
"io"
"log"
"log/syslog"
Expand All @@ -16,6 +17,10 @@ import (
"github.com/skycoin/skycoin/src/util/logging"
"github.com/spf13/cobra"

"net/http"
_ "net/http/pprof" //no_lint

"github.com/pkg/profile"
"github.com/skycoin/skywire/pkg/node"
"github.com/skycoin/skywire/pkg/util/pathutil"
)
Expand All @@ -27,13 +32,35 @@ var (
syslogAddr string
tag string
cfgFromStdin bool
profileMode string
pport string
)

var rootCmd = &cobra.Command{
Use: "skywire-node [config-path]",
Short: "App Node for skywire",
Run: func(_ *cobra.Command, args []string) {

profilePath := profile.ProfilePath("./logs/" + tag)
switch profileMode {
case "cpu":
defer profile.Start(profilePath, profile.CPUProfile).Stop()
case "mem":
defer profile.Start(profilePath, profile.MemProfile).Stop()
case "mutex":
defer profile.Start(profilePath, profile.MutexProfile).Stop()
case "block":
defer profile.Start(profilePath, profile.BlockProfile).Stop()
case "trace":
defer profile.Start(profilePath, profile.TraceProfile).Stop()
case "http":
go func() {
log.Println(http.ListenAndServe(fmt.Sprintf("localhost:%v", pport), nil))
}()
default:
// do nothing
}

logger := logging.MustGetLogger(tag)

if syslogAddr != "none" {
Expand Down Expand Up @@ -102,6 +129,8 @@ func init() {
rootCmd.Flags().StringVarP(&syslogAddr, "syslog", "", "none", "syslog server address. E.g. localhost:514")
rootCmd.Flags().StringVarP(&tag, "tag", "", "skywire", "logging tag")
rootCmd.Flags().BoolVarP(&cfgFromStdin, "stdin", "i", false, "read config from STDIN")
rootCmd.Flags().StringVarP(&profileMode, "pprof", "p", "none", "enable profiling with pprof. Mode: none or one of: [cpu, mem, mutex, block, trace, http]")
rootCmd.Flags().StringVarP(&pport, "pport", "", "6060", "port for http-mode of pprof")
}

// Execute executes root CLI command.
Expand Down
5 changes: 4 additions & 1 deletion cmd/skywire-node/skywire-node.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@ skywire application node
*/
package main

import "github.com/skycoin/skywire/cmd/skywire-node/commands"
import(
"github.com/skycoin/skywire/cmd/skywire-node/commands"

)

func main() {
commands.Execute()
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ require (
github.com/mattn/go-colorable v0.1.1 // indirect
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect
github.com/mitchellh/go-homedir v1.1.0
github.com/pkg/profile v1.3.0
github.com/prometheus/client_golang v0.9.2
github.com/sirupsen/logrus v1.4.1
github.com/skycoin/skycoin v0.25.1
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1f
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/pkg/profile v1.3.0 h1:OQIvuDgm00gWVWGTf4m4mCt6W1/0YqU7Ntg0mySWgaI=
github.com/pkg/profile v1.3.0/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.9.2 h1:awm861/B8OKDd2I/6o1dy3ra4BamzKhYOiGItCeZ740=
Expand Down
10 changes: 10 additions & 0 deletions vendor/github.com/pkg/profile/.travis.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions vendor/github.com/pkg/profile/AUTHORS

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 24 additions & 0 deletions vendor/github.com/pkg/profile/LICENSE

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

54 changes: 54 additions & 0 deletions vendor/github.com/pkg/profile/README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions vendor/github.com/pkg/profile/mutex.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions vendor/github.com/pkg/profile/mutex17.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit fe56a51

Please sign in to comment.