Swarm nodes don't find each other

[petyaracz]

System information

Swarm
Version: 0.3.7-unstable
Go Version: go1.10.3
OS: darwin

Expected behaviour

Swarm nodes with the same network id find each other.

Actual behaviour

Sometimes nodes find each other. Sometimes they don't. In that case, sometimes they find each other in a few minutes. Sometimes they don't.

This is a behaviour that is challenging to replicate, even on my own machine.

Steps to reproduce the behaviour

To demonstrate the isue, I used the swarm test network which builds a network in docker using terraform.

But I'm sure I've seen this issue come up (a) outside this setup (b) outside docker.

echo 'did it connect?' > diditconnect.txt

terraform apply -var 'swarm_count=2'
yes
sleep 30
docker exec -it swarm1 geth --exec 'admin.peers' attach /data/bzzd.ipc | wc -l >> diditconnect.txt
terraform destroy
yes

I repeated lines 3-8 a bunch of times (because terraform </3 loops). The resulting file diditconnect.txt looks sg like this: 1,3,1,1,23,22,1,1,1
So no connection the 1st-4th time and so admin.peers returns [] (I have no idea what 3 is!). And then connection the 5th-6th time (so `admin.peers' returns node info on node 2) and then again no connection for 7th-9th.

[zelig]

filed under ethersphere ethersphere/swarm#1020