Issue using seeding script and refs-to-seed with hashes that have no seeds

[sroerick]

So I'm currently trying to set up a gateway to pin a larger list of content.

Using make seems to work OK, but the script hangs because I'm running into content that I don't think is seeded at all.

It would be nice to be able to identify which hashes are hanging, in order that we could either find them and seed them, or remove them from the original list.

Because the script is recursively seeding hashes, I'm unable to find which is the parent hash of the hashes that are hanging.

I'm not sure what the best way to approach fixing this is, perhaps a timeout in the seeding script or in ipfs pin?

Being able to pin recursively would also be useful.

[davidar]

Yeah, a couple of people have already come across that (myself included). I think @jbenet already fixed some of it, but obviously there's still some issues. From memory, the problematic hash was

# ipfs-geoip
/ipfs/QmQQ3BUpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB

Edit: also see ipfs/kubo#1630

[sroerick]

Ah, OK. so ipfs pin -r does work in this case. That helps a little bit.

As I said, I have my own list of hashes to seed, they aren't the ipfs ones.

but I can start working manually and using pin -r to try and identify it

[sroerick]

still running into pretty big problems with this.

it's a big challenge because there isn't really any feedback in the UI.

Ideally, it'd be great to say: ipfs pin -r [listofhashes], and then see something like:
adding 42 hashes, 237 recursive hashes

followed by a progress bar for each hash.

The issue is that i'm running into timeouts and the command is stalling. I don't know which hash is causing the issue, and I'm having to use trial and error to add each one by one.

[jbenet]

Hey @sroerick -- that sounds very bad, sorry, let's get your problems fixed.

Ideally, it'd be great to say: ipfs pin -r [listofhashes], and then see something like:
adding 42 hashes, 237 recursive hashes

followed by a progress bar for each hash.

this sounds great. agreed on making the UX better. maybe open an issue in go-ipfs with a detail description of the UX that you would love?

The issue is that i'm running into timeouts and the command is stalling. I don't know which hash is causing the issue, and I'm having to use trial and error to add each one by one.

One thing i often do is run ipfs refs -r <hash> beforehand. just to see them stream in. i also run bsdash. see https://github.com/jbenet/node-bsdash

[davidar]

👍 better pinning ux

[RichardLitt]

One thing i often do is run ipfs refs -r beforehand. just to see them stream in. i also run bsdash. see https://github.com/jbenet/node-bsdash

If you do it, it might be good to mention it somewhere in the program mans or incorporate your flow directly.

[jbenet]

agreed. we really should just improve the pinning UX overall, and not have to do refs.

[sroerick]

thanks @jbenet,

opened ipfs/kubo#1711

[ghost]

Please reopen if there's anything actionable here, from an infrastructure point-of-view