Support DNSLink resolution

[bbondy]

All of this works only when IPFS Resolve Method is not Disabled and not Ask.

• DNSLink lookup works only for pages with 5xx response codes without a check for x-ipfs-path header and according to the IPFS Resolve Method settings above

How DNSLink lookup works:

• it looks for a DNS TXT record of next format dnslink=[value] for _dnslink.[hostname] domain. It should meet current dns preferences and use DoH and Secure DNS according to the settings. DNS requests should be cached and work without sending second requests to DNS servers for same hostname.
• If the lookup is successful and the dnslink is present it shows the Open in IPFS button in location bar, the button navigates to ipns://[hostname]

For pages with other response codes we check for x-ipfs-path header in response and according to the IPFS Resolve Method settings we show the Open in IPFS button in location bar without lookups, the button navigates to ipns://[hostname]

We should not show the botton for IPFS Gateway urls.

Example TXT record:
dnslink=/ipfs/<CID for your content here>

[lidel]

Sweet! This will be fantastic onboarding experience and I feel we should do this as soon as we can.
(cc @autonome @jessicaschilling @Gozala)

tip: if you find it too expensive to run additional DNX TXT lookup for every visited website, you can optimize things by:

• doing the lookup only when x-ipfs-path HTTP header is present
  (so only if site announces that it's also available on IPFS)
• or if page returned code different from OK 200 or failed due to network/DNS (eg. invalid A record)
  (this will make it possible for people to recover from broken/DDoSs HTTP servers etc)

[bertrandfalguiere]

I think the DDoS use-case is very important.
I can see the headline in tech news after the next inevitable DNS DoS: "How your website could have been up when the Internet was down last Friday Step one: put a DNSLink to your website, etc...."

In parallel, onboarding of website should be made as easy as possible (but i'm a bit off-topic here).

[jessicaschilling]

Love it! Small nit: would Open with IPFS or Open using IPFS be more accurate?

[bbondy]

@jessicaschilling good call

[Gozala]

Would be nice if opt-in in here would be persisted in some way, so that next navigation would go straight to ipfs:// as opposed to prompting user once again. Otherwise I fear user needs to be highly motivated to reload already loaded page.

[bbondy]

@Gozala see here:
#13611

[spylogsster]

as decided in slack chat

• we have to check original domain + _dnslink.[domain] prefix for TXT records, for example,

dig +short TXT _dnslink.en.wikipedia-on-ipfs.org
"dnslink=/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco"

dig +short TXT brantly.eth.link
"dnslink=/ipfs/QmeXGvLAsLwuq4uDLqsH3XECWZYiJrAszRvkDuyR91ERQX"

• we should not do this lookup in private windows and tor windows

[diracdeltas]

@bbondy does this only show up for users who already have Redirect IPFS resources to the configured IPFS gateway enabled?

[bbondy]

@bbondy does this only show up for users who already have Redirect IPFS resources to the configured IPFS gateway enabled?

It would show up any time the x-ipfs-path header is present in a response body of the main frame.

[diracdeltas]

It would show up any time the x-ipfs-path header is present in a response body of the main frame.

i don't think this should be running for users who've already set IPFS to disabled. suggested that in the code review and it was added here: https://github.com/brave/brave-core/pull/8068/files#diff-b2b99579204c3baf1314de0455a345809f2fee17c392a9a8b612dd839e99be9dR147

for users whose setting is 'ask' (the default), i confirmed it shows the interstitial page when 'open in IPFS' is clicked. 👍

[bbondy]

I agree on the disabled state, good catch. Thanks!

[diracdeltas]

• or if page returned code different from OK 200 or failed due to network/DNS (eg. invalid A record)
  (this will make it possible for people to recover from broken/DDoSs HTTP servers etc)

it looks like the current implementation will initiate 2 dnslink lookups for every status code from 500-504 (https://github.com/brave/brave-core/pull/8068/files#diff-b2b99579204c3baf1314de0455a345809f2fee17c392a9a8b612dd839e99be9dR182-R185)

given the extra scrutiny on brave network requests lately, perhaps we should ask the user for consent to do these lookups if they aren't already opted into IPFS? for instance show a prompt similar to the one for the wayback machine.

[bbondy]

@diracdeltas @spylogsster @lidel how about so we can make progress here we limit to only when an IPFS selection is made away from Ask and not Disable. I.e. only for Gateway and Local options. We can discuss what to do with Ask in a different issue.

[lidel]

@bbondy sgtm.

• Let's unblock this for users who gave clear consent to enable IPFS ("Gateway" or "Local")
• As for "Ask" behavior, ok to add it later, I like the idea suggested by @diracdeltas that it could be a similar/same consent UI as for 404 errors and existing Wayback Machine integration.

[diracdeltas]

• Let's unblock this for users who gave clear consent to enable IPFS ("Gateway" or "Local")
• As for "Ask" behavior, ok to add it later, I like the idea suggested by @diracdeltas that it could be a similar/same consent UI as for 404 errors and existing Wayback Machine integration.

sgtm!

[spylogsster]

as discussed in slack with @lidel we should ignore dnslink value and the button should navigate to ipns://[domain], for example:
in the case when https://en.wikipedia-on-ipfs.org has TXT record dnslink=/ipfs/QmeXGvLAsLwuq4uDLqsH3XECWZYiJrAszRvkDuyR91ERQX we show in address bar ipns://en.wikipedia-on-ipfs.org and the button navigates to ipns://en.wikipedia-on-ipfs.org

[spylogsster]

Updated description to meet new conditions after some discussions in slack

[stephendonner]

Verified PASSED using nightly on macOS build

Brave	1.23.31 Chromium: 89.0.4389.86 (Official Build) nightly (x86_64)
Revision	0c2dac31bc21e806a791641316e99920f84ea52c-refs/branch-heads/4389@{#1472}
OS	macOS Version 11.2.3 (Build 20D91)

Verified:

• brantly.eth.link and en.wikipedia-on-ipfs.org load tabs with Open in IPFS on the right-hand side of the URL bar

screenshots

• when Automatically redirect to IPFS pages via DNSLink when possible is enabled, loading brantly.eth.link and en.wikipedia-on-ipfs.org loads ipns://en.wikipedia-on-ipfs.org/wiki/ and ipns://brantly.eth.link/#/ respectively

screenshots

• Learn more links to https://docs.ipfs.io/concepts/dnslink/
• brantly.eth.link and en.wikipedia-on-ipfs.org load tabs via New Private Window and New Private Window with Tor, but do not offer to Open in IPFS, nor do they; they only load their HTTPS links, https://brantly.eth.link/#/ and https://en.wikipedia-on-ipfs.org/wiki/, respectively, with Automatically redirect to IPFS pages via DNSLink when possible enabled

screenshots

Preffed off:

Preffed on:

I also verified that when both: 1) Method to resolve IPFS resources is Disabled and 2) Automatically redirect to IPFS pages via DNSLink when possible is also preffed off, we don't auto-resolve or offer to load IPFS resources, and the URLs are loaded over their original HTTPS counterparts

screenshots

[maxihatop]

I think, your current algorithm:

DNSLink lookup works only for pages with 5xx response codes

is not good, because of:

1. If I would like to use IPFS for deploy file, this is assume, I do not use web-server.
2. If I must use "fake web-server", always returns error 5xx, then this will drop decentralization of IPFS nature: whenever server blocked, there no way to send to Brave status code 501, and activate DNSLink. As result, blocked centralized server will block access to decentralized IPFS content.
3. More worst scenario - when server is seized, and start returns fake answers with code 200. In this case, Bravo will automatically switch to answers from that server, and ignore IPFS-files. Client unable to aware of such "upgrade", and as result, security of such approach reduced to security of centralized server.

I think, more nice and correct idea - to use IP-address 0.0.0.0 (INADDR_ANY) as a flag of "there is no server, use dnslink".
Just for example:

1. User create DNS-record "example.com", contains A-record as "A=0.0.0.0"
2. User creates subdomain "_dnslink.example.com", contains regular dnslink record, as same as in using now.
3. When browser receives URL, it anyway calls gethostbyname(). If it receives "A=0.0.0.0", it instantly start lookup DNSLink.
4. As result, this is quick and efficient algorithm for seamless using DNSLink in the browser.

As an alternative: instead of A=0.0.0.0 we can use 127.0.0.127. This address included into loopback netmask /8, and actually don't used widely. We can reuse it as a flag for "No site, search DNSLink"

[lidel]

There is no need for special convention around A record: if there is no HTTP server to return error 500, then there is a network error of some sort, and that should also recover via DNSLink if TXT record is present.

IPFS Companion explicitly listens for specific error events and triggers DNSLink lookup for recovery:

  'net::ERR_NAME_NOT_RESOLVED', // dns failure
  'net::ERR_CONNECTION_TIMED_OUT', // eg. http server is offline
  'net::ERR_INTERNET_DISCONNECTED' // no network

Brave could do the same (if it does not already).

Sidenote about threat model: if block occurs at DNS level, all bets are off, because the same entity that censored A record can remove TXT one. If you want to protect yourself from that level of threats, you should not use DNS, need to use cryptographic addresses like IPNS.

[Jeff-Bouchard]

There is no need for special convention around A record: if there is no HTTP server to return error 500, then there is a network error of some sort, and that should also recover via DNSLink if TXT record is present.

IPFS Companion explicitly listens for specific error events and triggers DNSLink lookup for recovery:

  'net::ERR_NAME_NOT_RESOLVED', // dns failure
  'net::ERR_CONNECTION_TIMED_OUT', // eg. http server is offline
  'net::ERR_INTERNET_DISCONNECTED' // no network

Brave could do the same (if it does not already).

Sidenote about threat model: if block occurs at DNS level, all bets are off, because the same entity that censored A record can remove TXT one. If you want to protect yourself from that level of threats, you should not use DNS, need to use cryptographic addresses like IPNS.

or use a Real decentralized DNS Server and there's only one at the moment that's RFC-1035 compliant: EmerDNS. The rest are glorified bookmark managers. no offense intended.

[kay-is]

I tried this after creating a DNSLink domain for a page, and it didn't work.

If I activate "Automatically redirect to IPFS pages via DNSLink when possible," it would lead to a "could not resolve name" error.

If I deactivated it, everything would work as normal, but then Brave had basically the same behavior as all other browsers, which makes its IPFS features kinda moot.

[maxihatop]

I assume, soemthing wrong on Brave side. Or, you do not use OpenNIC (or another emerDNS) resolver. I tested it with Mozilla IPFS plugin, everything works perfect, see the screenshot: http://olegh.ftp.sh/emercoin/ipfs-emerdns.png
And this is DNS record in the emerDNS blockchain: https://explorer.emercoin.com/nvs/dns/ipfs-docs.emc/