feat: Persist peers to disk such that nodes can bootstrap from previously seen peers

[Wondertan]

All our infrastructure is based around bootstrappers, and each time a node starts; it connects to them and only to them. We do not save any information about other discovered peers during runtime on disk.

This is a choke point for our networks. In case network bootstrappers go offline, a node connected to the network won't be able to rejoin it after the restart.

Saving known peers on disk and using them for bootstrapping will solve the problem, decrease the load on bootstrappers, and enhance network decentralization.

To ultimately achieve the goal, we should:

• Use persistent PeerStore instead of in memory
• Periodically saving some fraction of connected peers and using them for bootstrapping instead of hardcoded bootstrappers
  • For this, we should introduce a Bootstrapper p2p component that handles/contains the saving and reconnecting logic
  • We must implement this first and then migrate to libp2p native solutions once Persist routing table libp2p/go-libp2p-kad-dht#254 is implemented.
• The storing of discovered full nodes and the reconnection to them on start should be in the Discovery component. We should additionally make periodic rotations of discovered full nodes.
  • This point needs more thinking, though, and saving might be unnecessary if we do rotations anyway

[renaynay]

@derrandz is taking over this issue.

TODO (in order):

• Documenting peer-usage throughout celestia-node (specifically in the libs/header/p2p and share/p2p packages) that results in a doc.go per package

  • libs/header/p2p peer-usage documentation
  • share/p2p documentation

• Detailed design of proposed solution (results in mini-ADR / creating a tasklist on this issue -- mini ADR not necessary but preferable! 😄 )

  • ADR 14

• Implementation based on tasklist

  • Feat: Bootstrap from previously seen peers #2133
  • Feat: add peerstore go-header#36
  • Feat: add subjective init opts go-header#37