MV3 Collector Branch

[whizzzkid]

This is the Collector PR for MV3 Work

Closes: #666

This PR:

• Has all the work required to transition to MV3.
• Companion MV3 Plan - Roadmap #1152

[SgtPooki]

FYI, with the latest companion beta release, I tested going to https://ipfs.io/ipns/en.wikipedia-on-ipfs.org/wiki/, and it stayed on the page without refresh. When I reloaded, it's indefinitely loading and not redirecting the the companion set gateway:

[SgtPooki]

https://explore.ipld.io/#/explore/QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D also results in an infinite spinner. (and doesn't without companion enabled)

If I disable companion when the infinite spinner is spinning, it takes me to ipns://explore.ipld.io/#/explore/QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D and shows an error:

could not get block QmYqLv4mWA8JN55F7qdBMcdcrz9Dwyc7z5xVhGgb6h1vCW: promise channel was closed

[whizzzkid]

@SgtPooki Can you please:

1. go to chrome://extensions
2. click on inspect views: service worker
3. go to console in the inspector
4. Share the output of: chrome.declarativeNetRequest.getDynamicRules() (copy-paste that here)
5. Now delete the existing rules using this snippet:

(async () => await chrome.declarativeNetRequest.updateDynamicRules({ addRules: [], removeRuleIds: await chrome.declarativeNetRequest.getDynamicRules().then((rules) => rules.map((rule) => rule.id)) }))()

6. Try again?

[SgtPooki]

@whizzzkid

Share the output of: chrome.declarativeNetRequest.getDynamicRules() (copy-paste that here)

it's empty:

await chrome.declarativeNetRequest.getDynamicRules()
[]

[SgtPooki]

FYI that http://explore-ipld-io.ipns.localhost:48084/#/ which shows ipns://explore.ipld.io/#/explore/QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm in the browser, does load when I wait long enough.

http://en-wikipedia--on--ipfs-org.ipns.localhost:48084/wiki/Wikipedia also loads, showing ipns://en.wikipedia-on-ipfs.org/wiki/ in the browser.

await chrome.declarativeNetRequest.getDynamicRules() still returns an empty array though

After changing from the brave provided node to my local node, then loading a few pages and waiting a bit.. I am able to get some dynamic rules:

[
  {
    "action": {
      "redirect": {
        "regexSubstitution": "http://localhost:8080/ipns/en.wikipedia-on-ipfs.org\\1"
      },
      "type": "redirect"
    },
    "condition": {
      "excludedInitiatorDomains": [],
      "regexFilter": "^http\\:\\/\\/en-wikipedia--on--ipfs-org\\.ipns\\.localhost\\:48084((?:[^\\.]|$).*)$",
      "resourceTypes": [
        "csp_report",
        "font",
        "image",
        "main_frame",
        "media",
        "object",
        "other",
        "ping",
        "script",
        "stylesheet",
        "sub_frame",
        "webbundle",
        "websocket",
        "webtransport",
        "xmlhttprequest"
      ]
    },
    "id": 27402,
    "priority": 1
  },
  {
    "action": {
      "redirect": {
        "regexSubstitution": "http://localhost:8080\\1"
      },
      "type": "redirect"
    },
    "condition": {
      "excludedInitiatorDomains": [],
      "regexFilter": "^http\\:\\/\\/127\\.0\\.0\\.1\\:8080(.*)$",
      "resourceTypes": [
        "csp_report",
        "font",
        "image",
        "main_frame",
        "media",
        "object",
        "other",
        "ping",
        "script",
        "stylesheet",
        "sub_frame",
        "webbundle",
        "websocket",
        "webtransport",
        "xmlhttprequest"
      ]
    },
    "id": 21038,
    "priority": 1
  },
  {
    "action": {
      "redirect": {
        "regexSubstitution": "http://localhost:8080\\1"
      },
      "type": "redirect"
    },
    "condition": {
      "excludedInitiatorDomains": [],
      "regexFilter": "^http\\:\\/\\/\\[\\:\\:1\\]\\:8080(.*)$",
      "resourceTypes": [
        "csp_report",
        "font",
        "image",
        "main_frame",
        "media",
        "object",
        "other",
        "ping",
        "script",
        "stylesheet",
        "sub_frame",
        "webbundle",
        "websocket",
        "webtransport",
        "xmlhttprequest"
      ]
    },
    "id": 10116,
    "priority": 1
  }
]

[SgtPooki]

FYI: turning the node off does not turn off redirects.

Example: https://starbase.gw3.io/ipfs/QmawceGscqN4o8Y8Fv26UUmB454kn2bnkXV5tEQYc4jBd6 redirected me to http://e580f2bcfc4276d5.gw3.io/ipfs/QmawceGscqN4o8Y8Fv26UUmB454kn2bnkXV5tEQYc4jBd6?sargs=H4sIAAAJbogA%2F6pWKlCyyivNydFRyoAxEpWsoquVspWslBKL0nXz8lNSlXSUypSslJwdXYM9vYrdLBw9QvMT80ITjUqKfSKcc319Awwy%2FSMKfS3KkyPNXSNDIjzzMkM9y5RqdRDmlOZlVkDNMTSzMLY0MrUwMlGqja0FDADpVtIcgwAAAA%3D%3D&ssig=APidkWklmXsEYaiSP5iTdxnwdnp1zerguGM6veRAv6j3NILaCMm%2BlFKgueQbTF99pG6ihtquyosQtb4u4grzCA%3D%3D, and when I refreshed, I was redirected to http://bafybeif3ie6d3yf2orkshi6xahlmwqudxst6db7mefkw6rcwanxwsksqou.ipfs.localhost:8080/?sargs=H4sIAAAJbogA%2F6pWKlCyyivNydFRyoAxEpWsoquVspWslBKL0nXz8lNSlXSUypSslJwdXYM9vYrdLBw9QvMT80ITjUqKfSKcc319Awwy%2FSMKfS3KkyPNXSNDIjzzMkM9y5RqdRDmlOZlVkDNMTSzMLY0MrUwMlGqja0FDADpVtIcgwAAAA%3D%3D&ssig=APidkWklmXsEYaiSP5iTdxnwdnp1zerguGM6veRAv6j3NILaCMm%2BlFKgueQbTF99pG6ihtquyosQtb4u4grzCA%3D%3D.

I disabled companion, and it was still redirecting me. Dynamic rules:

[
  {
    "action": {
      "redirect": {
        "regexSubstitution": "http://localhost:8080\\1"
      },
      "type": "redirect"
    },
    "condition": {
      "excludedInitiatorDomains": [],
      "regexFilter": "^https?\\:\\/\\/starbase\\.gw3\\.io((?:[^\\.]|$).*)$",
      "resourceTypes": [
        "csp_report",
        "font",
        "image",
        "main_frame",
        "media",
        "object",
        "other",
        "ping",
        "script",
        "stylesheet",
        "sub_frame",
        "webbundle",
        "websocket",
        "webtransport",
        "xmlhttprequest"
      ]
    },
    "id": 19877,
    "priority": 1
  },
  {
    "action": {
      "redirect": {
        "regexSubstitution": "http://localhost:8080\\1"
      },
      "type": "redirect"
    },
    "condition": {
      "excludedInitiatorDomains": [],
      "regexFilter": "^http\\:\\/\\/e580f2bcfc4276d5\\.gw3\\.io((?:[^\\.]|$).*)$",
      "resourceTypes": [
        "csp_report",
        "font",
        "image",
        "main_frame",
        "media",
        "object",
        "other",
        "ping",
        "script",
        "stylesheet",
        "sub_frame",
        "webbundle",
        "websocket",
        "webtransport",
        "xmlhttprequest"
      ]
    },
    "id": 6047,
    "priority": 1
  }
]

[SgtPooki]

done with a little over half of the files changed. will come back to it after ipfs-desktop release

[SgtPooki]

if POSSIBLE_NODE_TYPES is set to all the possible node types, then POSSIBLE_NODE_TYPES.includes(ipfsNodeType) will always be true, meaning supportRedirectToCustomGateway will always be true, right? Do we need a separate CUSTOM_GATEWAY_SUPPORTING_NODE_TYPES or should we remove this flag and dependent logic?

[whizzzkid]

Yep and that's the intention, it's basically a reverse of the previous check as embedded node type does not exist anymore, but this is just to make sure if after migration someone has it set to embedded then it does not break, this can go away after.

[SgtPooki]

we should have a list of follow up tasks from MV3 then, in a tracking issue

[whizzzkid]

Added a section: #1152

[whizzzkid]

@SgtPooki responded to most of your comments, let me know if anything else comes up.

[lidel]

💭 @whizzzkid is this a bug or a coincidence that both Chrome and Firefox use v111 in respective manifests?
iirc Firefox shipped MV3 in v109, are we depending on some fixes that landed later?

[whizzzkid]

At the time I was using 111, it's just a coincidence. There was a bug in chrome at least where the service worker would go to sleep and won't wake. I don't think FF had any.

[SgtPooki]

done looking over all code again. Approved but with some comments

[SgtPooki]

nit: we should be able to make this more type-safe with conditional types, to reduce the chance of a run-time error

[SgtPooki]

I think being more clear will reduce the chance of bugs in the future, but i'm fine to defer this if there's a followup issue.

[SgtPooki]

Is there a communication bus we can use to remove the need for this arbitrary wait time?

[lidel]

@whizzzkid I think this should be solved/moved to notifyOptionChange or even sendMessageToSelf, perhaps? there is a comment about discrepancy between Chrome/Firefox there already.

We don't want random timeouts like that sprinkled in random places, will make debugging of future race conditions even harder.

[whizzzkid]

notifyOptionChange is already triggered on Line182, but that resolves arbitrarily (the next line does not always get that change)

This behaviour is also not consistent with what happens in chrome and firefox. I tried with sendMessageToSelf and that doesn't work always either.

The only reliable way to do this at the time was this. I can add a TODO: to evaluate this in future?

[SgtPooki]

Why aren't we persisting the rest of the state?

Suggested change

	await browser.storage.local.set({ active: state.active })
	await browser.storage.local.set({ ...state })

[whizzzkid]

This set method is slightly different than a normal Map.set method. Passing the entire object will update all the keys and make this more expensive. So just pass the keys that need updating is the idea here.

https://developer.chrome.com/docs/extensions/reference/storage/#type-StorageArea:~:text=to%20the%20callback.-,set,-void

[SgtPooki]

do we need to call await browser.storage.local.set({ active: state.active }) twice in this block?

[whizzzkid]

I think this should go, I'll remove it.

[SgtPooki]

nit: should the 'awaitFinalUrl` method could be moved into here?

[whizzzkid]

#1279

[lidel]

Thank you for your work and patience @whizzzkid, and for reviews @SgtPooki ! Huge lift, but important for long-term utility of this extension.

We have pretty good test coverage now, guarding the most important feature (redirects) in both Firefox and Chromium.
I've tested in Firefox, Brave, and generic Chromium: user-facing functionality works as expected. If we have bugs, those will be small ones around niche edge cases.

I think the rc/3.0-mv3 branch baked long enough.
My suggestion is to merge this PR and then do remaining fixes/refactors in follow-up PRs against the main branch:

• [MV3 Beta Bugs] Invalid "Target" in Redirect Rules UI #1277
• [MV3 Beta Bugs] Single catch-all rule per subdomain gateway #1278

[AntiMoron]

Folks, support me. protobufjs/protobuf.js#1941