IPFS pubsub stops working after a while

[pleerock]

• Version:

"ipfs-core": "^0.16.1"

• Platform:

I test this issue on both Apple Intel and Apple M1.
Also issue was reproduced in both node.js and browser.

Severity:

High

Description:

After x number of messages sent (I'm talking about publishing to the topics) pubsub stops receiving messages COMPLETELY.

The only thing that help is repo directory removal. In case of the browsers the only thing helps is clearing browser storage.

Steps to reproduce the error:

I've published a reproduction repo here.

[pleerock]

Ohhh after two days of research I was able to find a place where the problem might be.

So, if we reset a datastore in libp2p options, everything seem to work:

    const ipfsd = await create({
        repo: "./ipfs-user-2",
        libp2p: (options) => {
            return createLibp2p({
                ...options.libp2pOptions,
                datastore: null
            })
        },

However at this point I have no idea what datastore is for and what this change will affect. Need your help here.

[tabcat]

Seeing similar issue in node with ipfs-core version 0.17

gist: https://gist.github.com/tabcat/8fc26a03a58617ebea0e8ff4d261fcb3

Behavior seen:

when starting ipfs with an existing repo, peers do not become pubsub peers.

first time running, before repo exists and is then created, everything works great.
second time running, using existing repo, the two nodes do not become pubsub peers.

[tabcat]

This issue may be a duplicate of #4212

[marcus-pousette]

I think it is the same issue as #4212 I wrote a solution there that fixed (I think) the issue for me

[marcus-pousette]

Another reason for this problem is that you are running into limits with how many WebSockets connections you have in the browser, which is fairly limited. Each topic is basically its own connection.

Long shot but, if you get MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 message lis teners added. Use emitter.setMaxListeners() to increase limit you might run into problems. The default limit for Event Emitter is 10. You can increase it with the emitter.setMaxListeners. Try like emitter.setMaxListeners(100). See this

Lastly, try to make you IPFS config as minimal as possible and only connect to what you need to connect to, to prevent overhead in networking

[shatilov-diman]

The issue maybe linked to libp2p #1504 which has been fixed at v0.41.0.
Currently ipfs-core released with v0.40.0
Hope it will come soon

[achingbrain]

js-IPFS is in the process of being deprecated, the replacement is Helia - please see the State of IPFS in JS post for a bit of background and the migration guide for how to port your app over.

Helia incorporates the latest libp2p stack so please try it out and open an issue there or https://github.com/ChainSafe/js-libp2p-gossipsub if the problem persists.