Synchronisation failed, dropping peer - even from trusted local 2nd node

[SvenMeyer]

System information

Geth Version: 1.8.9-stable
Git Commit: ff9b146
OS & Version: Linux Mint 18.3
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.10
Operating System: linux
GOPATH=
GOROOT=/usr/lib/go-1.10

Expected behaviour

geth should sync and download new blocks block by block

Actual behaviour

Synchronisation failed, dropping peer

Steps to reproduce the behaviour

I have two nodes in my local net work

192.168.1.201
geth 1.8.9-stable
Lubuntu 18.04

192.168.1.202
geth 1.8.9-stable
Linux Mint 18.3

both nodes know each other and show up in the peer list
node ...201 is syncing fine
node ...202 produces a lot of "Synchronisation failed, dropping peer" message even for the definitely "not bad" other node ...201 (peer=81b95b9bc444563d)

INFO [05-29|15:32:54] Imported new chain segment               blocks=1 txs=171  mgas=7.987  elapsed=3.202s    mgasps=2.494  number=5695324 hash=e84fb2…928bd3 cache=83.13mB
INFO [05-29|15:32:56] Imported new chain segment               blocks=1 txs=211  mgas=7.934  elapsed=2.150s    mgasps=3.689  number=5695324 hash=e4d557…b552ad cache=83.45mB
INFO [05-29|15:32:59] Imported new chain segment               blocks=1 txs=176  mgas=7.995  elapsed=2.860s    mgasps=2.795  number=5695325 hash=b69abc…3738c5 cache=84.36mB
WARN [05-29|15:33:05] Synchronisation failed, retrying         err="block header download canceled (requested)"
WARN [05-29|15:33:15] Head headers broke chain ordering        peer=81b95b9bc444563d index=0 requested=5695133 received=5695338
WARN [05-29|15:33:15] Synchronisation failed, dropping peer    peer=81b95b9bc444563d err="retrieved hash chain is invalid"
WARN [05-29|15:33:38] Synchronisation failed, retrying         err="block download canceled (requested)"
WARN [05-29|15:33:41] Synchronisation failed, retrying         err="block download canceled (requested)"
WARN [05-29|15:33:56] Head headers broke chain ordering        peer=d38c9698a21f9226 index=0 requested=5695133 received=5695341
WARN [05-29|15:33:56] Synchronisation failed, dropping peer    peer=d38c9698a21f9226 err="retrieved hash chain is invalid"
WARN [05-29|15:34:22] Header broke chain ancestry              number=5695346 hash=1a2f2c…cd0a55
WARN [05-29|15:34:22] Synchronisation failed, dropping peer    peer=81b95b9bc444563d err="action from bad peer ignored"
WARN [05-29|15:36:17] Header broke chain ordering              number=5695349 hash=13ea8b…74757d expected=5695352
WARN [05-29|15:36:17] Synchronisation failed, dropping peer    peer=81b95b9bc444563d err="action from bad peer ignored"
INFO [05-29|15:37:42] Imported new chain segment               blocks=5 txs=855  mgas=35.517 elapsed=10.187s   mgasps=3.486  number=5695330 hash=00c7a2…909c16 cache=88.17mB
INFO [05-29|15:37:52] Imported new chain segment               blocks=4 txs=875  mgas=31.955 elapsed=10.029s   mgasps=3.186  number=5695334 hash=e03a38…9e255c cache=91.87mB
INFO [05-29|15:38:00] Imported new chain segment               blocks=5 txs=558  mgas=31.677 elapsed=8.101s    mgasps=3.910  number=5695339 hash=6e6b9c…6d8f5a cache=94.53mB
INFO [05-29|15:38:09] Imported new chain segment               blocks=5 txs=822  mgas=33.882 elapsed=9.226s    mgasps=3.672  number=5695344 hash=11dbe9…2ac361 cache=98.33mB
INFO [05-29|15:38:18] Imported new chain segment               blocks=6 txs=540  mgas=47.915 elapsed=9.476s    mgasps=5.056  number=5695350 hash=dee586…57d398 cache=102.35mB
INFO [05-29|15:38:26] Imported new chain segment               blocks=3 txs=450  mgas=17.431 elapsed=7.224s    mgasps=2.413  number=5695353 hash=0a52de…058082 cache=103.83mB
INFO [05-29|15:38:27] Imported new chain segment               blocks=1 txs=315  mgas=7.926  elapsed=1.719s    mgasps=4.610  number=5695332 hash=e09e8b…cd2b0b cache=103.96mB
WARN [05-29|15:39:33] Head headers broke chain ordering        peer=81b95b9bc444563d index=0 requested=5695161 received=5695365
WARN [05-29|15:39:33] Synchronisation failed, dropping peer    peer=81b95b9bc444563d err="retrieved hash chain is invalid"
WARN [05-29|15:41:18] Synchronisation failed, dropping peer    peer=81b95b9bc444563d err="retrieved hash chain is invalid"
WARN [05-29|15:41:27] Synchronisation failed, dropping peer    peer=bd8632dbbf116d4a err="action from bad peer ignored"
WARN [05-29|15:42:24] Header broke chain ordering              number=5695377 hash=6a48c6…542b46 expected=5695378
WARN [05-29|15:42:24] Synchronisation failed, dropping peer    peer=15a4480e7ac71425 err="action from bad peer ignored"
WARN [05-29|15:43:07] Head headers broke chain ordering        peer=5b65f4152dced902 index=0 requested=5695161 received=5695384
WARN [05-29|15:43:07] Synchronisation failed, dropping peer    peer=5b65f4152dced902 err="retrieved hash chain is invalid"

{
    caps: ["eth/62", "eth/63", "les/1", "les/2"],
    id: "81b95b9bc444563d3defb4cf3b1e66308d8ce9f2e072562325f596c82a03f1d349e57dfe93736afacac4428fe9b55b8a3ce6b9b19a1ec70d34aab21931d31178",
    name: "Geth/v1.8.9-stable-ff9b1461/linux-amd64/go1.10.1",
    network: {
      inbound: false,
      localAddress: "192.168.1.202:53480",
      remoteAddress: "192.168.1.201:30303",
      static: true,
      trusted: false
    },
    protocols: {
      eth: {
        difficulty: 4.4599500175833893e+21,
        head: "0x138d7403f54cb4da5224d9c58c01a08d2ad8f783ba8e183da6378582c94aae10",
        version: 63
      }
    }
}

Backtrace

[backtrace]

[locdinh209]

I got the same issue @SvenMeyer. Geth server usually get "Synchronisation failed, dropping peer" and make server sync slow. I also got it more after upgrade server to v1.8.9-stable.

[keysonZZZ]

I got same problem with one geth node after I upgrade from v1.8.8-stable to v1.8.9-stable.

System information

Geth Version: 1.8.9-stable
Git Commit: ff9b146
OS & Version: Ubuntu14.04
Architecture: amd64
Go Version: go1.10

console logs

INFO [05-30|07:25:44] Imported new chain segment               blocks=1  txs=154  mgas=7.990   elapsed=738.886ms mgasps=10.814 number=5701450 hash=0bdbf7…f7f82c cache=67.15mB
INFO [05-30|07:25:45] Imported new chain segment               blocks=1  txs=95   mgas=7.999   elapsed=355.739ms mgasps=22.485 number=5701451 hash=337045…f709ea cache=67.84mB
INFO [05-30|07:25:45] Imported new chain segment               blocks=1  txs=157  mgas=8.003   elapsed=857.748ms mgasps=9.330  number=5701452 hash=9fe11a…3625d6 cache=68.43mB
INFO [05-30|07:25:46] Imported new chain segment               blocks=1  txs=107  mgas=8.006   elapsed=393.543ms mgasps=20.342 number=5701453 hash=01c7f2…0a5d2d cache=69.25mB
WARN [05-30|07:26:27] Synchronisation failed, dropping peer    peer=76405b65434379f9 err=timeout
WARN [05-30|07:27:13] Synchronisation failed, dropping peer    peer=1ff5fc918f9d96a2 err=timeout
WARN [05-30|07:27:57] Synchronisation failed, dropping peer    peer=acc10cb785abbaa0 err=timeout
WARN [05-30|07:28:42] Synchronisation failed, dropping peer    peer=ee9acd73547fb856 err=timeout
WARN [05-30|07:29:28] Synchronisation failed, dropping peer    peer=ae6f9d4f849b43ce err=timeout
WARN [05-30|07:30:17] Synchronisation failed, dropping peer    peer=1118980bf48b0a36 err=timeout
WARN [05-30|07:31:03] Synchronisation failed, dropping peer    peer=d7f18f41098a3a5e err=timeout
WARN [05-30|07:31:48] Synchronisation failed, dropping peer    peer=ead21049df30c0e9 err=timeout
WARN [05-30|07:32:34] Synchronisation failed, dropping peer    peer=01686be14cf6d96b err=timeout
WARN [05-30|07:33:18] Synchronisation failed, dropping peer    peer=ae95c795259a6215 err=timeout
WARN [05-30|07:34:03] Synchronisation failed, dropping peer    peer=313689b05d7d978d err=timeout
WARN [05-30|07:34:48] Synchronisation failed, dropping peer    peer=316f5ddf95ceb041 err=timeout
WARN [05-30|07:35:33] Synchronisation failed, dropping peer    peer=766ac72775b24d3f err=timeout
WARN [05-30|07:36:18] Synchronisation failed, dropping peer    peer=4a78d657edcfc918 err=timeout
WARN [05-30|07:39:29] Synchronisation failed, dropping peer    peer=8e02330a1c6cebf4 err=timeout
WARN [05-30|07:40:14] Synchronisation failed, dropping peer    peer=d5ac62e9d0644372 err=timeout
WARN [05-30|07:41:01] Synchronisation failed, dropping peer    peer=ea6a667263e51598 err=timeout
WARN [05-30|07:45:27] Synchronisation failed, dropping peer    peer=9638573ca791dc03 err=timeout

and the geth console get stuck.When I try to restart the geth command using ctrl +C.The geth get stuck on INFO [05-30|07:57:08] Stopping Ethereum protocol for more than ten minutes.After I restart geth process. It worked well in a while.Then still drop peers like this:

INFO [05-30|08:20:46] Imported new chain segment               blocks=1  txs=122  mgas=7.991   elapsed=290.970ms mgasps=27.462 number=5701659 hash=3bea87…50b8b8 cache=147.27mB
INFO [05-30|08:20:52] Imported new chain segment               blocks=1  txs=268  mgas=7.995   elapsed=314.463ms mgasps=25.425 number=5701660 hash=8781d3…a334c1 cache=148.14mB
WARN [05-30|08:20:55] Synchronisation failed, dropping peer    peer=219949d17cf2e5cd err=timeout
INFO [05-30|08:21:04] Imported new chain segment               blocks=1  txs=169  mgas=7.988   elapsed=284.780ms mgasps=28.050 number=5701658 hash=3a4a8e…32dcd2 cache=148.28mB
INFO [05-30|08:21:04] Imported new chain segment               blocks=1  txs=343  mgas=8.001   elapsed=394.217ms mgasps=20.295 number=5701661 hash=ade1c7…8a4a22 cache=148.52mB
INFO [05-30|08:21:32] Imported new chain segment               blocks=1  txs=348  mgas=7.996   elapsed=382.439ms mgasps=20.908 number=5701662 hash=aa3789…c96b9a cache=149.26mB
WARN [05-30|08:32:34] Synchronisation failed, retrying         err="header processing canceled (requested)"
WARN [05-30|08:33:11] Synchronisation failed, dropping peer    peer=316f5ddf95ceb041 err=timeout
WARN [05-30|08:33:47] Synchronisation failed, dropping peer    peer=ea6a667263e51598 err=timeout
WARN [05-30|08:34:25] Synchronisation failed, dropping peer    peer=ae6f9d4f849b43ce err=timeout
WARN [05-30|08:35:01] Synchronisation failed, dropping peer    peer=829c606175d3c75b err=timeout

[keysonZZZ]

I upgrade to v1.8.10-stable and everything seems to be okay now.

[locdinh209]

@keysonZZZ I have already updated to v1.8.10-stable but it got the same issues in v1.8.9-stable :(.

WARN [05-31|01:08:58] Synchronisation failed, dropping peer peer=32f71984da7bd5da err="no peers available or all tried for download"
WARN [05-31|01:09:33] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:10:40] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
INFO [05-31|01:12:39] Imported new chain segment blocks=2 txs=197 mgas=15.977 elapsed=2.050s mgasps=7.791 number=5705469 hash=c741b7…5b1c0e cache=107.74mB
INFO [05-31|01:12:41] Imported new chain segment blocks=2 txs=290 mgas=15.973 elapsed=1.930s mgasps=8.272 number=5705471 hash=68c114…bdfec1 cache=108.75mB
WARN [05-31|01:12:45] Synchronisation failed, dropping peer peer=32f71984da7bd5da err="no peers available or all tried for download"
INFO [05-31|01:13:21] Imported new chain segment blocks=2 txs=140 mgas=15.983 elapsed=1.055s mgasps=15.140 number=5705473 hash=48a7e7…d0b65c cache=109.31mB
INFO [05-31|01:13:24] Imported new chain segment blocks=2 txs=234 mgas=15.979 elapsed=2.039s mgasps=7.835 number=5705475 hash=31f8bd…b1cdb0 cache=109.92mB
INFO [05-31|01:13:26] Imported new chain segment blocks=2 txs=327 mgas=15.967 elapsed=1.950s mgasps=8.187 number=5705477 hash=27d6c7…baf564 cache=110.59mB
INFO [05-31|01:13:28] Imported new chain segment blocks=2 txs=259 mgas=15.965 elapsed=1.832s mgasps=8.713 number=5705479 hash=fc7e9e…6ddd94 cache=111.55mB
WARN [05-31|01:13:31] Synchronisation failed, dropping peer peer=32f71984da7bd5da err="no peers available or all tried for download"
WARN [05-31|01:16:02] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:17:30] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:18:41] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:23:41] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:24:13] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:25:01] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:26:50] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
INFO [05-31|01:27:44] Imported new chain segment blocks=2 txs=321 mgas=15.977 elapsed=7.853s mgasps=2.035 number=5705481 hash=9b6a93…c69c1d cache=112.16mB
INFO [05-31|01:27:51] Imported new chain segment blocks=4 txs=346 mgas=31.940 elapsed=6.224s mgasps=5.131 number=5705485 hash=57a420…c9212e cache=113.44mB
WARN [05-31|01:27:51] Synchronisation failed, dropping peer peer=32f71984da7bd5da err="no peers available or all tried for download"
WARN [05-31|01:30:23] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|01:37:11] Synchronisation failed, dropping peer peer=b4fafe2b18e6a0a1 err=timeout
INFO [05-31|01:59:51] Regenerated local transaction journal transactions=2 accounts=1
WARN [05-31|02:00:40] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|02:01:31] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
INFO [05-31|02:03:21] Imported new chain segment blocks=2 txs=159 mgas=15.960 elapsed=1.430s mgasps=11.159 number=5705487 hash=96786a…2b9975 cache=113.62mB
WARN [05-31|02:03:26] Synchronisation failed, retrying err="block body download canceled (requested)"
WARN [05-31|02:04:41] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|02:05:50] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|02:09:51] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|02:10:23] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout
WARN [05-31|02:10:53] Synchronisation failed, dropping peer peer=32f71984da7bd5da err=timeout

It "Synchronisation failed, dropping peer" many time and make sync block very slow

[SvenMeyer]

I have the same experience. Upgraded both my local nodes to 1.8.10, still the nodes produce timeouts, are often 20 - 30 blocks behind.
The interesting observation I have made is: If I block port 30303 on my router, so that the local nodes are not discoverable from the outside, first the have less peers but also after a while of stabilization and catching up with the latest blocks, the error and time-out rate drops considerably - give it a try.
Next step would why the open connection to the outside world causes such problems.

[locdinh209]

@SvenMeyer I think it will make the server catching up with the latest blocks. But why should we need to open connection to the outside again, could you explain more about it, pls?
P/s: After you open connection again, is the error come back ?

[SvenMeyer]

@pk209 (as far as I understand), if you open the 30303 port to the outside world, then other nodes can also discover your node. In that case the number of nodes mine was connected to also jumped from about ~ 5 to about 45 ! ... I also enabled to accept (up to 2) LightClients, which increased the upload data volume by quite at lot and made the node even more unstable ....

[Cryptome2]

Same problem here. Can't get Geth stable since one month. Constantly peer timeouts and sync faillures. Tried different servers, all versions of Geth .

[Atrides]

Confirm the issue. On one of my nodes. Hangs on block 5756748, now actual block 5757090.
All ports open, node has only pure geth running.

WARN [06-09|12:22:42] Synchronisation failed, dropping peer    peer=4dfd6d9ac021bd48 err=timeout
WARN [06-09|12:23:33] Synchronisation failed, dropping peer    peer=c14de4d7f9d386a3 err=timeout
WARN [06-09|12:24:22] Synchronisation failed, dropping peer    peer=40e95335eb63d282 err=timeout
WARN [06-09|12:25:13] Synchronisation failed, dropping peer    peer=39e3434772d7517d err=timeout
WARN [06-09|12:26:02] Synchronisation failed, dropping peer    peer=d36011c11ec4bf6e err=timeout
WARN [06-09|12:26:53] Synchronisation failed, dropping peer    peer=de18e8b88d1e6f1b err=timeout
WARN [06-09|12:27:43] Synchronisation failed, dropping peer    peer=760860494af47180 err=timeout
WARN [06-09|12:28:19] Synchronisation failed, retrying         err="block download canceled (requested)"
INFO [06-09|12:28:25] Commit new mining work                   number=5756749 txs=70   uncles=0 elapsed=9m57.517s
WARN [06-09|12:29:09] Synchronisation failed, dropping peer    peer=14a13faf9178abf7 err=timeout

> WARN [06-09|12:29:58] Synchronisation failed, dropping peer    peer=0111e5a4a4c2bc7d err=timeout
> eth.syncing
{
  currentBlock: 5756748,
  highestBlock: 5756896,
  knownStates: 151542401,
  pulledStates: 151542401,
  startingBlock: 5756455
}
WARN [06-09|12:36:27] Synchronisation failed, retrying         err="block download canceled (requested)"
WARN [06-09|12:37:16] Synchronisation failed, dropping peer    peer=f6fa1156d37bafe5 err=timeout

[Atrides]

Daemon stopped and deleted all except for folder with blockchain, only [ /nodes/, transactions.rlp, /ethash/ ], the daemon works. Then after some hours hangs again.
UPD: helps if deleted only transactions.rlp

[prashantprabhakar]

is there a solution to this issue. Seems this is in the queue for long and no one has a reason on How to resolve this. Mostly restarting geth does the trick, but it's lot manual work.

The transaction drops peers every now and then and blockchain is out of sync.

[acidfreako]

after waiting for a while my node continue to sync 1.8.11

[alexfisher]

Having this issue on 1.8.11-stable. Restarted a couple times. Now, it just hangs on IPC endpoint opened.

[sgyyz]

As right now the only solution to bring it back is restarting the nodes.

So is it possible to setup 2 geth nodes(Server 01 and Server 02) and protect the nodes in a private network like the temp solution, connect 01 and 02. And then expose 01 as your mainly work node for application, on the contrast, server 02 used to make sure it sync the latest blocks by automatically restart in 30 mins or etc. Then the server 01 can sync the latest block from your server 02.

[Cryptome2]

I gave it up. Dead end. I can't get Geth stable last 2 months. Tried different servers, different locations in the world , tried it several times. Nodes will sync first and but after a few minutes they will start dropping peers and failing sync. Don't understand why the Geth developers can't make this stable.

[SvenMeyer]

I gave up as well. I do not understand if/how other people manage to get the geth nodes running stable. I recently upgraded to 1.8.12, but as soon as the node catches up to the lasted block, then the problems start: "Synchronisation failed, dropping peer" , even with 2 nodes within the same network.

[raybotha]

I have three nodes running on the same network, even they fall out of sync with each other with 100+ blocks occasionally...

I've had some luck the last day or so with blocking port 30303 ingress on nodes you need to fetch very recent data from, we'll have to see how this goes though.

[taoxinyi]

I've had this similar problem err="retrieved hash chain is invalid" when I try to sync two nodes on different machines, and it turns out the system time on the machines is not the same.
DEBUG[08-29|12:38:41.560] Downloaded item processing failed number=67 hash=b85a23…51f618 err="future block: 1535517555 > 1535517551".
In fact, A's time is 4 seconds later than B's, so that B cannot import from A.
After change their system time to the same, they can sync well.

[bestplay]

1.8.17
Same issue

[cdljsj]

1.8.17
Same issue
Have to restart the node to bypass it

[ngkong]

is #18072 related to this issue?

[BlinkyStitt]

I think I'm seeing this issue.

$ geth version
Geth
Version: 1.8.20-stable
Git Commit: 24d727b6d6e2c0cde222fa12155c4a6db5caaf2e
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.10.4
Operating System: linux
GOPATH=
GOROOT=/usr/lib/go-1.10

INFO [12-22|19:55:50.979] Imported new chain segment               blocks=1  txs=116 mgas=5.031  elapsed=8.272s    mgasps=0.608 number=6934267 hash=f92970…e9cf4b age=9m2s      cache=136.49mB
INFO [12-22|19:55:58.247] Imported new chain segment               blocks=2  txs=15  mgas=14.212 elapsed=7.267s    mgasps=1.955 number=6934269 hash=c36b84…606bc2 age=8m41s     cache=136.06mB
WARN [12-22|19:55:58.247] Synchronisation failed, dropping peer    peer=66d66a505191f82d err="action from bad peer ignored"
WARN [12-22|19:56:11.798] Synchronisation failed, dropping peer    peer=45f4c318af8108c7 err=timeout
WARN [12-22|19:56:21.737] Synchronisation failed, dropping peer    peer=4a8bc79176509fbc err=timeout
WARN [12-22|19:59:45.533] Synchronisation failed, dropping peer    peer=7d2368f2817b7427 err="action from bad peer ignored"
WARN [12-22|20:00:32.592] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
WARN [12-22|20:02:03.735] Synchronisation failed, dropping peer    peer=710b8f1e18232104 err=timeout
WARN [12-22|20:03:52.592] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
WARN [12-22|20:04:21.928] Synchronisation failed, dropping peer    peer=b6d6aa0cfb4792cf err=timeout
WARN [12-22|20:06:31.928] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
WARN [12-22|20:07:53.735] Synchronisation failed, dropping peer    peer=305c675798d59b24 err=timeout
WARN [12-22|20:08:02.318] Synchronisation failed, dropping peer    peer=5d28afae14b210f0 err=timeout
INFO [12-22|20:11:40.152] Imported new chain segment               blocks=1  txs=131 mgas=7.979  elapsed=11.898s   mgasps=0.671 number=6934270 hash=cbf2bc…a63c10 age=23m55s    cache=136.48mB
INFO [12-22|20:11:45.380] Imported new chain segment               blocks=1  txs=33  mgas=7.189  elapsed=5.228s    mgasps=1.375 number=6934271 hash=e2c013…2779bd age=23m46s    cache=136.53mB

Then it happened again 20 minutes later.

INFO [12-22|20:31:44.697] Imported new chain segment               blocks=3  txs=115 mgas=9.302  elapsed=8.318s    mgasps=1.118 number=6934417 hash=3f0a21…e59afc age=5m43s     cache=78.92mB
INFO [12-22|20:31:54.897] Imported new chain segment               blocks=1  txs=38  mgas=7.422  elapsed=10.199s   mgasps=0.728 number=6934418 hash=37b6bf…b48ffe age=5m49s     cache=79.46mB
INFO [12-22|20:32:08.762] Imported new chain segment               blocks=3  txs=124 mgas=10.614 elapsed=13.864s   mgasps=0.766 number=6934421 hash=a7bf9f…fbb218 age=5m37s     cache=79.46mB
INFO [12-22|20:32:31.900] Imported new chain segment               blocks=4  txs=223 mgas=18.403 elapsed=23.138s   mgasps=0.795 number=6934425 hash=1c9525…8f061c age=5m1s      cache=80.45mB
WARN [12-22|20:32:41.735] Synchronisation failed, dropping peer    peer=65536ce61e77d8a7 err=timeout
INFO [12-22|20:36:35.577] Regenerated local transaction journal    transactions=0 accounts=0
WARN [12-22|20:38:12.280] Synchronisation failed, dropping peer    peer=a2810181f9b03201 err=timeout
WARN [12-22|20:40:17.735] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
WARN [12-22|20:41:51.928] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
WARN [12-22|20:45:11.996] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
WARN [12-22|20:46:14.735] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
INFO [12-22|20:50:00.073] Imported new chain segment               blocks=1  txs=96  mgas=6.726  elapsed=11.457s   mgasps=0.587 number=6934426 hash=f887c6…6fcdd0 age=22m27s    cache=81.03mB

And again right now:

INFO [12-22|21:30:07.465] Imported new chain segment               blocks=1  txs=32  mgas=7.936  elapsed=13.827s   mgasps=0.574 number=6934649 hash=8eeb87…2e948c age=8m        cache=129.40mB
INFO [12-22|21:30:21.176] Imported new chain segment               blocks=2  txs=150 mgas=10.877 elapsed=13.711s   mgasps=0.793 number=6934651 hash=8676a4…b9b2f1 age=7m39s     cache=130.13mB
WARN [12-22|21:30:21.177] Synchronisation failed, dropping peer    peer=e045ecb05eed9e53 err="action from bad peer ignored"
WARN [12-22|21:32:35.704] Synchronisation failed, dropping peer    peer=655d36ecac72b461 err=timeout
WARN [12-22|21:35:47.428] Synchronisation failed, retrying         err="block download canceled (requested)"
INFO [12-22|21:36:42.599] Regenerated local transaction journal    transactions=0 accounts=0
WARN [12-22|21:36:56.467] Synchronisation failed, retrying         err="block download canceled (requested)"

Closing port 30303 might help, but that doesn't seem like a good suggestion for the network as a whole.

EDIT: Actually, I just checked and port 30303 is closed on my node.

[kkuehm]

Constant issue. Managed to see Geth spit out a combination of "time out" and "block download canceled" (mostly timeout) for 5 hours solid before I intervened with a reset.

[holiman]

This ticket is stale -- the p2p networking is changed from time to time, and we there's no real info we can use here to investigate. The last issue from @marcosmartinez7 is unrelated (Clique deadlock), and from @wysenynja -- that looks good, if it drops peers sometimes but continues syncing it's not an issue, it's to be expected.

[BarneyChambers]

I am still seeing this issue at random. My node has run fine for about a month, but now suddenly I cannot find peers.

[nikitasius]

1.8.27 same.

seriosly....

[BlinkyStitt]

Sorry for the delay. @holiman, when my node drops peers, it stops syncing. I restart it manually and it starts catching up right away. There is definitely a bug here that needs investigating.

I don’t check my node every day so I don’t usually notice until I see my server CPU load is lower than normal. Then I’ll check and see that the latest block is like 3 days behind.

I’m contemplating just checking the node every hour with a cron script and restarting it. Not very graceful, but it should work

[C0ntr07]

Check that the system time is correct. If the node's clock is materially off then the node will not synchronize or function properly.

... at least that is my experience.

[nikitasius]

Check that the system time is correct. If the node's clock is materially off then the node will not synchronize or function properly.

... at least that is my experience.

time is always automatically synced with ntp

[abenmariem]

In windows, when i made time sync automatically from windows server, the error is fixed and i managed to stake