Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify protocol: SignedPeerRecords not being added to the CertifiedAddrBook #2754

Closed
Tracked by #2704
PedrobyJoao opened this issue Mar 26, 2024 · 5 comments · Fixed by libp2p/go-libp2p-pubsub#555
Assignees
Labels
kind/bug A bug in existing code (including security flaws) P0 Critical: Tackled by core team ASAP

Comments

@PedrobyJoao
Copy link

PedrobyJoao commented Mar 26, 2024

Version Information
github.com/PedrobyJoao/koko
cloud.google.com/go v0.37.0
dmitri.shuralyov.com/app/changes v0.0.0-20180602232624-0a106ad413e3
dmitri.shuralyov.com/html/belt v0.0.0-20180602232347-f7d459c86be0
dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412
dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c
git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96
github.com/BurntSushi/toml v0.3.1
github.com/alecthomas/kingpin/v2 v2.4.0
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
github.com/benbjohnson/clock v1.3.5
github.com/beorn7/perks v1.0.1
github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625
github.com/buger/jsonparser v0.0.0-20181115193947-bf1c66bbce23
github.com/cespare/xxhash v1.1.0
github.com/cespare/xxhash/v2 v2.2.0
github.com/chromedp/cdproto v0.0.0-20230802225258-3cf4e6d46a89
github.com/chromedp/chromedp v0.9.2
github.com/chromedp/sysutil v1.0.0
github.com/chzyer/readline v1.5.1
github.com/cilium/ebpf v0.9.1
github.com/client9/misspell v0.3.4
github.com/containerd/cgroups v1.1.0
github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d
github.com/coreos/go-systemd/v22 v22.5.0
github.com/cpuguy83/go-md2man/v2 v2.0.0
github.com/davecgh/go-spew v1.1.1
github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c
github.com/decred/dcrd/crypto/blake256 v1.0.1
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0
github.com/dgraph-io/badger v1.6.2
github.com/dgraph-io/ristretto v0.0.2
github.com/docker/go-units v0.5.0
github.com/dustin/go-humanize v1.0.0
github.com/elastic/gosigar v0.14.2
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
github.com/flynn/noise v1.1.0
github.com/francoispqt/gojay v1.2.13
github.com/fsnotify/fsnotify v1.5.4
github.com/ghodss/yaml v1.0.0
github.com/gliderlabs/ssh v0.1.1
github.com/go-errors/errors v1.0.1
github.com/go-kit/log v0.2.1
github.com/go-logfmt/logfmt v0.5.1
github.com/go-logr/logr v1.3.0
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572
github.com/gobwas/httphead v0.1.0
github.com/gobwas/pool v0.2.1
github.com/gobwas/ws v1.2.1
github.com/godbus/dbus/v5 v5.1.0
github.com/gogo/protobuf v1.3.2
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/golang/lint v0.0.0-20180702182130-06c8688daad7
github.com/golang/mock v1.2.0
github.com/golang/protobuf v1.5.3
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c
github.com/google/go-cmp v0.6.0
github.com/google/go-github v17.0.0+incompatible
github.com/google/go-querystring v1.0.0
github.com/google/gopacket v1.1.19
github.com/google/martian v2.1.0+incompatible
github.com/google/pprof v0.0.0-20240207164012-fb44976bdcd5
github.com/google/uuid v1.4.0
github.com/googleapis/gax-go v2.0.0+incompatible
github.com/googleapis/gax-go/v2 v2.0.3
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1
github.com/gorilla/mux v1.8.0
github.com/gorilla/websocket v1.5.1
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7
github.com/grpc-ecosystem/grpc-gateway v1.5.0
github.com/hashicorp/golang-lru/arc/v2 v2.0.5
github.com/hashicorp/golang-lru/v2 v2.0.5
github.com/huin/goupnp v1.3.0
github.com/ianlancetaylor/demangle v0.0.0-20230524184225-eabc099b10ab
github.com/ipfs/go-cid v0.4.1
github.com/ipfs/go-datastore v0.6.0
github.com/ipfs/go-ds-badger v0.3.0
github.com/ipfs/go-ds-leveldb v0.5.0
github.com/ipfs/go-log/v2 v2.5.1
github.com/jackpal/go-nat-pmp v1.0.2
github.com/jbenet/go-temp-err-catcher v0.1.0
github.com/jbenet/goprocess v0.1.4
github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1
github.com/jinzhu/inflection v1.0.0
github.com/jinzhu/now v1.1.5
github.com/joho/godotenv v1.5.1
github.com/josharian/intern v1.0.0
github.com/jpillora/backoff v1.0.0
github.com/json-iterator/go v1.1.12
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024
github.com/julienschmidt/httprouter v1.3.0
github.com/kisielk/errcheck v1.5.0
github.com/kisielk/gotool v1.0.0
github.com/klauspost/compress v1.17.6
github.com/klauspost/cpuid/v2 v2.2.7
github.com/koron/go-ssdp v0.0.4
github.com/kr/pretty v0.3.1
github.com/kr/pty v1.1.3
github.com/kr/text v0.2.0
github.com/libp2p/go-buffer-pool v0.1.0
github.com/libp2p/go-flow-metrics v0.1.0
github.com/libp2p/go-libp2p v0.33.0
github.com/libp2p/go-libp2p-asn-util v0.4.1
github.com/libp2p/go-libp2p-pubsub v0.10.0
github.com/libp2p/go-libp2p-testing v0.12.0
github.com/libp2p/go-msgio v0.3.0
github.com/libp2p/go-nat v0.2.0
github.com/libp2p/go-netroute v0.2.1
github.com/libp2p/go-reuseport v0.4.0
github.com/libp2p/go-yamux/v4 v4.0.1
github.com/libp2p/zeroconf/v2 v2.2.0
github.com/lunixbochs/vtclean v1.0.0
github.com/mailru/easyjson v0.7.7
github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd
github.com/mattn/go-isatty v0.0.20
github.com/mattn/go-sqlite3 v1.14.17
github.com/matttproud/golang_protobuf_extensions v1.0.4
github.com/microcosm-cc/bluemonday v1.0.1
github.com/miekg/dns v1.1.58
github.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c
github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b
github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1
github.com/minio/sha256-simd v1.0.1
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
github.com/modern-go/reflect2 v1.0.2
github.com/mr-tron/base58 v1.2.0
github.com/multiformats/go-base32 v0.1.0
github.com/multiformats/go-base36 v0.2.0
github.com/multiformats/go-multiaddr v0.12.2
github.com/multiformats/go-multiaddr-dns v0.3.1
github.com/multiformats/go-multiaddr-fmt v0.1.0
github.com/multiformats/go-multibase v0.2.0
github.com/multiformats/go-multicodec v0.9.0
github.com/multiformats/go-multihash v0.2.3
github.com/multiformats/go-multistream v0.5.0
github.com/multiformats/go-varint v0.0.7
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86
github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab
github.com/onsi/ginkgo/v2 v2.15.0
github.com/onsi/gomega v1.30.0
github.com/opencontainers/runtime-spec v1.2.0
github.com/openzipkin/zipkin-go v0.1.1
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58
github.com/pion/datachannel v1.5.5
github.com/pion/dtls/v2 v2.2.8
github.com/pion/ice/v2 v2.3.11
github.com/pion/interceptor v0.1.25
github.com/pion/logging v0.2.2
github.com/pion/mdns v0.0.9
github.com/pion/randutil v0.1.0
github.com/pion/rtcp v1.2.13
github.com/pion/rtp v1.8.3
github.com/pion/sctp v1.8.9
github.com/pion/sdp/v3 v3.0.6
github.com/pion/srtp/v2 v2.0.18
github.com/pion/stun v0.6.1
github.com/pion/transport/v2 v2.2.4
github.com/pion/turn/v2 v2.1.4
github.com/pion/webrtc/v3 v3.2.23
github.com/pkg/errors v0.9.1
github.com/pmezard/go-difflib v1.0.0
github.com/prometheus/client_golang v1.19.0
github.com/prometheus/client_model v0.6.0
github.com/prometheus/common v0.48.0
github.com/prometheus/procfs v0.12.0
github.com/quic-go/qpack v0.4.0
github.com/quic-go/qtls-go1-20 v0.3.4
github.com/quic-go/quic-go v0.41.0
github.com/quic-go/webtransport-go v0.6.0
github.com/raulk/go-watchdog v1.3.0
github.com/rogpeppe/go-internal v1.10.0
github.com/russross/blackfriday v1.5.2
github.com/russross/blackfriday/v2 v2.0.1
github.com/sergi/go-diff v1.0.0
github.com/shurcooL/component v0.0.0-20170202220835-f88ec8f54cc4
github.com/shurcooL/events v0.0.0-20181021180414-410e4ca65f48
github.com/shurcooL/github_flavored_markdown v0.0.0-20181002035957-2122de532470
github.com/shurcooL/go v0.0.0-20180423040247-9e1955d9fb6e
github.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041
github.com/shurcooL/gofontwoff v0.0.0-20180329035133-29b52fc0a18d
github.com/shurcooL/gopherjslib v0.0.0-20160914041154-feb6d3990c2c
github.com/shurcooL/highlight_diff v0.0.0-20170515013008-09bb4053de1b
github.com/shurcooL/highlight_go v0.0.0-20181028180052-98c3abbbae20
github.com/shurcooL/home v0.0.0-20181020052607-80b7ffcb30f9
github.com/shurcooL/htmlg v0.0.0-20170918183704-d01228ac9e50
github.com/shurcooL/httperror v0.0.0-20170206035902-86b7830d14cc
github.com/shurcooL/httpfs v0.0.0-20171119174359-809beceb2371
github.com/shurcooL/httpgzip v0.0.0-20180522190206-b1c53ac65af9
github.com/shurcooL/issues v0.0.0-20181008053335-6292fdc1e191
github.com/shurcooL/issuesapp v0.0.0-20180602232740-048589ce2241
github.com/shurcooL/notifications v0.0.0-20181007000457-627ab5aea122
github.com/shurcooL/octicon v0.0.0-20181028054416-fa4f57f9efb2
github.com/shurcooL/reactions v0.0.0-20181006231557-f2e0b4ca5b82
github.com/shurcooL/sanitized_anchor_name v1.0.0
github.com/shurcooL/users v0.0.0-20180125191416-49c67e49c537
github.com/shurcooL/webdavfs v0.0.0-20170829043945-18c3829fa133
github.com/sirupsen/logrus v1.8.1
github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d
github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e
github.com/spaolacci/murmur3 v1.1.0
github.com/stretchr/objx v0.1.0
github.com/stretchr/testify v1.8.4
github.com/syndtr/goleveldb v1.0.0
github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07
github.com/urfave/cli v1.22.2
github.com/viant/assertly v0.4.8
github.com/viant/toolbox v0.24.0
github.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7
github.com/xhit/go-str2duration/v2 v2.1.0
github.com/yuin/goldmark v1.4.13
go.opencensus.io v0.18.0
go.uber.org/atomic v1.11.0
go.uber.org/dig v1.17.1
go.uber.org/fx v1.20.1
go.uber.org/goleak v1.3.0
go.uber.org/mock v0.4.0
go.uber.org/multierr v1.11.0
go.uber.org/zap v1.27.0
go4.org v0.0.0-20180809161055-417644f6feb5
golang.org/x/build v0.0.0-20190111050920-041ab4dc3f9d
golang.org/x/crypto v0.19.0
golang.org/x/exp v0.0.0-20240213143201-ec583247a57a
golang.org/x/lint v0.0.0-20200302205851-738671d3881b
golang.org/x/mod v0.15.0
golang.org/x/net v0.21.0
golang.org/x/oauth2 v0.16.0
golang.org/x/perf v0.0.0-20180704124530-6e6d33e29852
golang.org/x/sync v0.6.0
golang.org/x/sys v0.17.0
golang.org/x/telemetry v0.0.0-20240208230135-b75ee8823808
golang.org/x/term v0.17.0
golang.org/x/text v0.14.0
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
golang.org/x/tools v0.18.0
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
google.golang.org/api v0.1.0
google.golang.org/appengine v1.6.7
google.golang.org/genproto v0.0.0-20190306203927-b5d61aea6440
google.golang.org/grpc v1.19.0
google.golang.org/protobuf v1.33.0
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
gopkg.in/inf.v0 v0.9.1
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1
gorm.io/driver/sqlite v1.5.5
gorm.io/gorm v1.25.8
grpc.go4.org v0.0.0-20170609214715-11d0a25b4919
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a
lukechampine.com/blake3 v1.2.1
sourcegraph.com/sourcegraph/go-diff v0.5.0
sourcegraph.com/sqs/pbtypes v0.0.0-20180604144634-d3ebe8f20ae4

Hey! I have been trying to use Gossipsub Peer Exchange (PX) feature for discovery/routing purposes, however when automatically trying to connect to the discovered peers (based on PX done by bootstrap peers), the same error happens for all new connection tries (for the returned peers):

2024-03-26T08:53:25.568-0300 DEBUG pubsub [email protected]/gossipsub.go:966 error connecting to QmeYAxZaJ9C3Tw8yp6XNopaJP2ctyrNysReJjhzFeiASJt: failed to dial: failed to dial QmeYAxZaJ9C3Tw8yp6XNopaJP2ctyrNysReJjhzFeiASJt: no addresses

After investigating a little bit, I saw that addresses will only be send within PX if they come from SignedPeerRecords.

I didn't know anything about SignedPeerRecords so I supposed I had to create and send them manually, until I realized that they were being automatically created when instantiating basicHost and being sent within the identify family of protocols.

I had setup a basic network for tests where there is one bootstrap peers and n other normal peers, they're all connecting to the bootstrap peer only (star topology). For debugging purposes, I added the following that runs every 30 seconds:

for _, p := range h.Peerstore().Peers() {
   rec := cab.GetPeerRecord(p)
   if rec == nil {
   	zlog.Sugar().Errorf("Peer %s has NO signed peer record", p)
   } else {
	zlog.Sugar().Debugf("Peer %s HAS signed peer record", p)
   }
}

Output in a nutshell: the host peer has only information about its own signed peer record! While for all the other connected peers, Peer has NO signed peer record is logged.

Why is that happening: Identify protocol

When receiving messages through the Identify protocols, the peer records are indeed processed BUT they are not added to the CertifiedAddrBook. See:

func (ids *idService) consumeMessage(mes *pb.Identify, c network.Conn, isPush bool) {
       ...
    	var addrs []ma.Multiaddr
	if signedPeerRecord != nil {
		signedAddrs, err := ids.consumeSignedPeerRecord(c.RemotePeer(), signedPeerRecord)
		if err != nil {
			log.Debugf("failed to consume signed peer record: %s", err)
		} else {
			addrs = signedAddrs
		}
	} else {
		addrs = lmaddrs
	}
       ...
}

func (ids *idService) consumeSignedPeerRecord(p peer.ID, signedPeerRecord *record.Envelope) ([]ma.Multiaddr, error) {
	if signedPeerRecord.PublicKey == nil {
		return nil, errors.New("missing pubkey")
	}
	id, err := peer.IDFromPublicKey(signedPeerRecord.PublicKey)
	if err != nil {
		return nil, fmt.Errorf("failed to derive peer ID: %s", err)
	}
	if id != p {
		return nil, fmt.Errorf("received signed peer record envelope for unexpected peer ID. expected %s, got %s", p, id)
	}
	r, err := signedPeerRecord.Record()
	if err != nil {
		return nil, fmt.Errorf("failed to obtain record: %w", err)
	}
	rec, ok := r.(*peer.PeerRecord)
	if !ok {
		return nil, errors.New("not a peer record")
	}
	if rec.PeerID != p {
		return nil, fmt.Errorf("received signed peer record for unexpected peer ID. expected %s, got %s", p, rec.PeerID)
	}
	// Don't put the signed peer record into the peer store.
	// They're not used anywhere.
	// All we care about are the addresses.
	return rec.Addrs, nil
}

It seems that is the expected behavior?

// Don't put the signed peer record into the peer store.
// They're not used anywhere.
// All we care about are the addresses.

So my question is: is this really an expected behavior?

What is the utility of processing SignedPeerRecords within the Identify protocols if they're being treated as normal unsigned records? Applications can not differentiate if the received listening addresses from Identify are signed or unsigned currently.

@PedrobyJoao
Copy link
Author

If that is the expected behavior, how should applications handle the use of Gossipsub PX with []multiaddr being shared? Do apps have to build another protocol just to exchange signed peer records?

@PedrobyJoao
Copy link
Author

PedrobyJoao commented Mar 26, 2024

Not sure yet if it's the expected behavior but I confirm that was the problem for my case, just forked the code and modified consumeMessage() to add the signedPeerRecord to the CertifiedAddrBook.

Now every peer has the signed records of every peers and PeerExchange (PX) is working without any other discovery method as the signed records are being shared with PX (and before being shared through PX, they are shared through identify)

Code modification here: PedrobyJoao@9504ce7

@sukunrt
Copy link
Member

sukunrt commented Mar 26, 2024

I think it's fine to add the signed peer record to the CertifiedAddrBook and remove it when the peer disconnects.

We should specify this though. Currently the identify spec says nothing about signed peer records: https://github.com/libp2p/specs/blob/master/identify/README.md

@sukunrt sukunrt self-assigned this Mar 28, 2024
@p-shahi p-shahi added kind/bug A bug in existing code (including security flaws) P1 High: Likely tackled by core team if no one steps up P0 Critical: Tackled by core team ASAP and removed P1 High: Likely tackled by core team if no one steps up labels Mar 28, 2024
@p-shahi p-shahi mentioned this issue Mar 28, 2024
22 tasks
@MarcoPolo
Copy link
Collaborator

First off, thank you @PedrobyJoao . This is a great example of a well crafted issue and bug report. I appreciate it!

The quick fix is certainly to start putting things back in the certified address book. But here are some reasons against it:

  1. go-libp2p itself doesn't use the certified address book. It seems a bit error prone to support and expose something we don't use. Contrast this to the event bus, which is used in many places and exposed to users.
  2. It relies on undocumented behavior of Identify reaching into the host's CertifiedAddressBook and updating it.
  3. We don't know what kind of properties protocols are relying on in this CertifiedAddressBook:
  • Should entries be removed on disconnect? You very well might want to keep these around for a bit depending on the application.
  • Should we only insert records that are signed by the peer we are connected to? Or can a peer give us a signed peer record of another peer? Again, depending on the application, both are valid.

I don't think the solution is to have Identify be responsible for updating the certified address book.

Here's my suggestion on how to fix it:

  1. Let's have Identify emit an event that includes the signed peer record.
  2. Let's then update gossipsub to subscribe to those events and keep track of them in a way that makes sense to it.
  • We can discuss on that PR what properties GossipSub expects from the CertifiedAddressBook. I'm not sure off the top of my head.

Even without 2, applications would still be able to update the CertifiedAddressBook themselves from events emitted by Identify.

It would be even better if we could allow go-libp2p to run modular components/services (as mentioned in #1993). Then if multiple protocols needed a certain kind of certified address book, they could construct it and share the same resource, but this can happen later.

@MarcoPolo
Copy link
Collaborator

Following up here to give a quick update:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A bug in existing code (including security flaws) P0 Critical: Tackled by core team ASAP
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants