Bezel hub crashed in swarm

[rasulovk]

Hi, I have issue running beszel hub in docker swarm, also I use glusterfs to share data between swarm nodes. But unfortunately, sometimes get crash of bezel hub container:

beszel_beszel.1.tep0hdefosur@no | database/sql.OpenDB.gowrap1()
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/database/sql/sql.go:833 +0x30 fp=0x40003fdfd0 sp=0x40003fdfa0 pc=0x166c90
beszel_beszel.1.tep0hdefosur@nod | runtime.goexit({})
beszel_beszel.1.tep0hdefosur@no | /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x40003fdfd0 sp=0x40003fdfd0 pc=0x88654
beszel_beszel.1.kba690a7az5i@nod | goroutine 12 gp=0x4000104e00 m=nil [select]:
beszel_beszel.1.tep0hdefosur@nod | created by database/sql.OpenDB in goroutine 1
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/database/sql/sql.go:833 +0x128
beszel_beszel.1.tep0hdefosur@nod |
beszel_beszel.1.kba690a7az5i@nod | runtime.gopark(0x400012bf78?, 0x2?, 0x70?, 0x0?, 0x400012bf74?)
beszel_beszel.1.tep0hdefosur@nod | goroutine 9 gp=0x400031ea80 m=nil [select]:
beszel_beszel.1.tep0hdefosur@nod | runtime.gopark(0x400006df78?, 0x2?, 0x70?, 0x0?, 0x400006df74?)
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/runtime/proc.go:424 +0xc8 fp=0x4000283e00 sp=0x4000283de0 pc=0x80348
beszel_beszel.1.kba690a7az5i@nod | /usr/local/go/src/runtime/proc.go:424 +0xc8 fp=0x4000133e00 sp=0x4000133de0 pc=0x80348
beszel_beszel.1.tep0hdefosur@nod | runtime.selectgo(0x4000283f78, 0x400006df70, 0x0?, 0x0, 0x0?, 0x1)
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/runtime/select.go:335 +0x67c fp=0x4000283f30 sp=0x4000283e00 pc=0x5dccc
beszel_beszel.1.tep0hdefosur@nod | database/sql.(*DB).connectionOpener(0x400013eb60, {0x19cba20, 0x400013b9f0})
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/database/sql/sql.go:1253 +0x80 fp=0x4000283fa0 sp=0x4000283f30 pc=0x168a90
beszel_beszel.1.tep0hdefosur@nod | database/sql.OpenDB.gowrap1()
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/database/sql/sql.go:833 +0x30 fp=0x4000283fd0 sp=0x4000283fa0 pc=0x166c90
beszel_beszel.1.tep0hdefosur@nod | runtime.goexit({})
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x4000283fd0 sp=0x4000283fd0 pc=0x88654
beszel_beszel.1.tep0hdefosur@nod | created by database/sql.OpenDB in goroutine 1
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/database/sql/sql.go:833 +0x128
beszel_beszel.1.tep0hdefosur@nod |
beszel_beszel.1.tep0hdefosur@nod | goroutine 10 gp=0x400031ec40 m=nil [select]:
beszel_beszel.1.tep0hdefosur@nod | runtime.gopark(0x40000707a8?, 0x2?, 0x0?, 0x0?, 0x400007078c?)
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/runtime/proc.go:424 +0xc8 fp=0x40000abe20 sp=0x40000abe00 pc=0x80348
beszel_beszel.1.tep0hdefosur@nod | runtime.selectgo(0x40000abfa8, 0x4000070788, 0x0?, 0x0, 0x0?, 0x1)
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/runtime/select.go:335 +0x67c fp=0x40000abf50 sp=0x40000abe20 pc=0x5dccc
beszel_beszel.1.tep0hdefosur@nod | github.com/pocketbase/pocketbase/core.(*BaseApp).initLogger.func3()
beszel_beszel.1.tep0hdefosur@nod | /go/pkg/mod/github.com/pocketbase/[email protected]/core/base.go:1351 +0x84 fp=0x40000abfd0 sp=0x40000abf50 pc=0xc14cc4
beszel_beszel.1.tep0hdefosur@nod | runtime.goexit({})
beszel_beszel.1.tep0hdefosur@nod | /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x40000abfd0 sp=0x40000abfd0 pc=0x88654
beszel_beszel.1.tep0hdefosur@nod | created by github.com/pocketbase/pocketbase/core.(*BaseApp).initLogger in goroutine 1
beszel_beszel.1.tep0hdefosur@nod | /go/pkg/mod/github.com/pocketbase/[email protected]/core/base.go:1347 +0x2c4

[rasulovk]

Client: Docker Engine - Community
Version: 27.4.1
API version: 1.47
Go version: go1.22.10
Git commit: b9d17ea
Built: Tue Dec 17 15:46:22 2024
OS/Arch: linux/arm64
Context: default

Server: Docker Engine - Community
Engine:
Version: 27.4.1
API version: 1.47 (minimum version 1.24)
Go version: go1.22.10
Git commit: c710b88
Built: Tue Dec 17 15:46:22 2024
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.7.24
GitCommit: 88bf19b2105c8b17560993bee28a01ddc2f97182
runc:
Version: 1.2.2
GitCommit: v1.2.2-0-g7cb3632
docker-init:
Version: 0.19.0
GitCommit: de40ad0

[henrygd]

Are you running on multiple nodes at the same time?

PocketBase, and Beszel hub by extension, should not be run in parallel due to lack of synchronization between instances.

If you need failover, it should be active/passive, where only one instance runs at a time, and a backup takes over only if the active instance fails.

I'm not a swarm expert, but I think you may be able to do this with replicas: 1. I'll do more testing with swarm when I have time.