Meta: window.ipfs v2

[lidel]

(This is a placeholder issue, will be updated in future)

The goal: window.ipfs v2 ships and is production ready
(tl;dr: injected on-demand, expose only requested APIs)

TL;DR

Future-proof way of using window.ipfs is:

if (window.ipfs) {
  try {
    const ipfs = await window.ipfs.enable({commands: ['add','cat']})
    const [{ hash }] = await ipfs.add(Buffer.from('=^.^='))
    const data = await ipfs.cat(hash)
    console.log(data.toString()) // =^.^=
  } catch (err) {
    // find more code samples at https://git.io/fjAu8
...

Implementation Status

#619:

• scaffolding for window.ipfs.enable
• decide if display prompt for .enable() (no params) and keep no-dialog for things like ipfs.cat or display no initial dialog but prompt every api instead
  • A: no change for now, logic stays the same
• Implement Bulk Permission Prompt at window.ipfs.enable (Add window.ipfs.enable(opts) (Bulk Permission Prompt) #619)
  • window.ipfs.enable({commands: ['id','version'] })
  • update docs
  • tests
• UX+API for requesting permissions in bulk
  • UI remains the same to keep this PR small
• add support for future opt-in experiments
  • separate namespace for optional experimentation without any stability guarantees
  • PoC await window.ipfs.enable({ experiments: { ipfsx: true } }) will return ipfsx version of the API
• add deprecation warning to API calls executed on window.ipfs

Open Questions

• Should we rename it to window.ipfsProxy.enable? (to avoid breaking websites that manually detect window.ipfs and execute window.ipfs.<command>) or is it enough to deprecate old use for 3 months and then remove support for old ways?
  • We will keep the name and go with deprecation period (2019-04-01), as noted in Add window.ipfs.enable(opts) (Bulk Permission Prompt) #619 (comment)
• now that we have window.ipfs.enable should we require permission from user for all commands? (remove acl-whitelist for things like dag.get ?)
  • We will remove whitelist, and nothing will be allowed without explicit permission from the user (rationale: Add window.ipfs.enable(opts) (Bulk Permission Prompt) #619 (comment))
• Check if it is feasible to disable window.ipfs injection via manifest in Chromium
  • now that we have async window.ipfs.enable we should revisit
  • context why we did it in the first place: window.ipfs injected after page loaded #362 (comment)

Remaining TODOs (separate PRs)

• thin content script, ipfs proxy client is lazy-loaded
  • stop exposing methods on window.ipfs
  • minimize the size of content script responsible for window.ipfs
  • lazy-init IPFS Proxy client on first call to window.ipfs.enable()
• improve UX of permission dialog
• remove Buffer and add support for Uint8Array/ArrayBuffer natively – RFC: Support ArrayBuffer as an alternative to Buffer input argument ipfs-inactive/interface-js-ipfs-core#404
• standardize error messages to follow js-ipfs/js-ipfs-http-client (after discussion in [WIP] feat: add errors with codes and utils js-ipfs#1746 is finalized and merged)
• A wrapper ("ipfs provider") library exists and understands "window.ipfs 2.0"
  • it should try ipfsCompanion.ipfs and fallback to js-ipfs by default
  • it should be possible to inject config to customize fallback heuristics and use js-ipfs-api with a specific URL before or instead falling back to js-ipfs
  • update window.ipfs-fallback
    • consider renaming to 'ipfs-api-provider' (or creating a new lib, or integrate with peer-base)
• Update /docs/window.ipfs.md
• Create/update three demo apps requesting different APIs via window.ipfs 2.0
• Publish a blog post about window.ipfs 2.0 at http://blog.ipfs.io and update old one to point at the updated one
• update known docs referring old window.ipfs, ensure they support window.ipfs 2.0
• Close all issues in related milestone

Design Notes

• Learn from MetaMask changes (more info)

  Rationale: MetaMask and IPFS Companion are often used by the same websites. To make things easier for web developers, we should try to follow conventions established by their implementation where possible and change them only if it does not translate well to our needs.

  • they changed approach and will continue polluting window object
  • by default there will be no identifiable data apart from the presence of window.ethereum attribute
  • async call to window.ethereum.enable() triggers permission dialog
    • when accepted, account details are exposed via window.ethereum
• Pick the safest design for "window.ipfs" 2.0
  • A: on-demand window.postMessage API injection scheme
    (thin content-script registers listener for window.postMessage + window.addEventListener)
    • 👍 no mutation of window object (assumption: there are no other fingerprinting vectors)
    • app asks for specific namespaces, user accepts/denies, api object is returned with only those namespaces
      • there should be no response on deny eliminating sniffing/fingerprinting
        • 💢 this introduces UX issue: how dapp can detect the browser supports ipfs without introducing artificial delay?
        • 💢 if dapp can detect the ipfs capability, malicious js running on the same page will as well. It is basically the same surface as having window.ipfs attribute. Given this reality, there is no value in over-complicating the detection.
  • B: inject a thin window.ipfs that has only a .enable() method
    • 👍 global flag on window solves the problem of signaling ipfs capability to dapp
    • API itself is never exposed as a global attribute: window.ipfs becomes a thin entry point
      • 👍 app asks for specific namespaces via async call to the window.ipfs.enable() method, user accepts/denies, actual API object is returned with only those namespaces, permission is cached and user is not prompted next time
      • 👍 we can return correct "access denied" response as it is no longer a surface for fingerprinting
      • await window.ipfs.enable(<capabilities>) should return a complete IPFS API instance
        • Calling an instance method that was not listed in <capabilities> should trigger ad-hoc permission prompt for that single capability (and update ACLs for the scope)
        • Calling window.ipfs.enable(<capabilities>) multiple times should be possible as well as an alternative, UX-friendly way to extend preexisting permissions with additional ones
        • ACLs are shared by all instances (or make instance itself a singleton per scope path)
    • 💢 Mutates window but there is only one additional bit of info malicious parties can use for fingerprinting.
      • It should be acceptable in the long term, if the goal is mass adoption, then fingerprinting based on using a browser with thin window.ipfs would be like fingerprinting based on if the user’s browser has notification or webcam possibilities. (related pro | con for Ethereum)
    • 👍 shipping this in backward-compatible way is easy, we could implement everything and deprecate direct calls to window.ipfs.* later
      • Deprecation Period: calling a method on a thin instance could be proxied transparently into calling .enable(arg) and then target method on a cached API object + printing warning in logs
      • Cutoff: accept only .enable() call
    • Global integration toggle should remove window.ipfs in Chrome
      • Using window.ipfs in synchronous fashion needs to be deprecated as it won't be fixed in Chrome.
      • We should educate users on good async/await practices and/or provide API provider library instead.
  • Common
    • include insight from Mozilla on the generic way vendors could support protocol-specific RPC-like APIs
    • requests for sensitive API namespaces (eg. config) should trigger more prominent visual warning (within the same, single dialog)

Additional References

Browser Vendors

• Fingerprinting
  • W3C: Mitigating Browser Fingerprinting in Web Specifications
  • Fingerprinting-Protection-Mode in Brave
• Relevant bugs
  • Chrome WONTFIX: The absence of synchronous message API make impossible to pass options to scripts that are loaded before the page to block content. (this means content script injected by webextension is unable to make a synchronous query to window before page loads)
  • Firefox Bug 1496510
    Turn on the behavior from bug 1329324 (disallowing defining non-configurable properties on a WindowProxy) on beta/release

Ethereum's window.ethereum

• Announcement: blog post
• Spec draft: eip-1102
• Main Discussion: EIP-1102: Opt-in provider access
  • Why navigator.permissions.* can't be used
  • Why window.postMessage can't respond with a full API provider
  • emit an event instead of post a response message, return the provider object to the event listener instead of injecting it globally
  • Twist: Rethinking total privacy

    DOM environments expose a read-only provider until full provider access is approved by the user.

  • New problem: multiple extensions injecting (and thus overwriting) providers
• Sister Discussion: The UX of EIP 1102
  • a crude but robust detection scheme: inject an iframe that refreshes until it detects web3
  • UX flow visualization of different approaches (total privacy, thin provides etc)

Places to Update (after the switch)

• https://github.com/tableflip/window.ipfs-fallback
• https://github.com/ipfs-shipyard/window.ipfs-is-required
• third-party libs like 1, 2
• Textile Build - Session 2 @ 57:46 – a simple annotation with a link to our blog post should be enough
• https://github.com/arxiu/ipld-mindmap#document-todos

[alanshaw]

Sounds like B is a good candidate for moving forwards.

actual API object is returned with only those namespaces, permission is cached and user is not prompted next time

Should we consider the case where an app would like basic permissions (low barrier to entry, less chance of hitting no)? 99% of the time the app only needs those permissions but very occasionally might want to do something more sensitive that requires additional permissions.

With the proposal above there's no way to "get more", or could you call .enable() multiple times - I guess so. Maybe something to note down! (sorry just brain dumping...)

It would be nicer to just be able to use the API and get a prompt when you try to use restricted API calls you haven't already requested access to. It means you don't have to code around missing properties and calling enable again and secondly, if you add some new feature and forget to request a permission you don't break your app.

Also, when I say "nicer" I think it might be necessary! The issue is if we have a provider library we don't know what type of ipfs we're getting given - we should just be able to use it like a regular IPFS instance.

I think we discussed a request permissions API once apon a time, what if we had window.ipfs.permissions.request() for requesting multiple permissions in one shot and just exposed the full API as we do now? The API is public knowledge, what additional attack surface area are we exposing by this being present? Is the only reason to not do this the performance impact of injecting the whole API?

[lidel]

Very good feedback!

I think we could do all of that without adding additional APIs:
just allowing multiple calls to .enable() and returning a "full" IPFS API instance should do the trick.

Added notes:

• await window.ipfs.enable(<capabilities>) should return a complete IPFS API instance
  • Calling an instance method that was not listed in <capabilities> should trigger ad-hoc permission prompt for that single capability (and update ACLs for the scope)
• Calling window.ipfs.enable(<capabilities>) multiple times should be possible as well
  • This is an alternative, UX-friendly way to extend pre-existing permissions with additional ones
  • ACLs are shared by all instances (we probably will just make instance itself a singleton per scope path)

Having that, a dapp first asks for basic dag get/put APIs (that triggers a calm user prompt,) and in rare case when dapp wants access to ipfs.config API a separate call can be issued to trigger a more prominent user prompt.

The API is public knowledge, what additional attack surface area are we exposing by this being present? Is the only reason to not do this the performance impact of injecting the whole API?

Yes, we don't want to load ipfs-postmsg-proxy nor initialize messaging port with background extension until window.ipfs.enable() is executed.

[lidel]

PR with await window.ipfs.enable(opts) aka Bulk Permission Prompt is ready for review at #619.

I also updated #589 (comment) to include remaining work plan.

[lidel]

window.ipfs.enable landed in Beta: v2.6.3.12520

tl;dr

• await window.ipfs.enable({ commands: ['id','peers'] }) triggers the Bulk Permission Prompt:

  

• deprecation warning is shown for window.ipfs.<cmd>():

  Calling commands directly on window.ipfs is deprecated and will be removed on 2019-04-01. Use API instance returned by window.ipfs.enable() instead. More: /docs/window.ipfs.md

  • Tip: where possible, use window.ipfs-fallback library that takes care of fallback ceremony.
    It will ensure your app follows API changes and does not break in the future.
• support opt-in ipfsx experiment (b633eb4)

  if experiments:{ipfsx:true} is passed to window.ipfs.enable it will return IPFS API instance wrapped in ipfsx prototype from ipfs-shipyard/ipfsx

  let ipfsx = await window.ipfs.enable({
     commands: ['add','files.addPullStream'],
     experiments: { ipfsx: true }
  })

• remove ACL whitelist for window.ipfs (756b177)

  no command can be run without explicit approval from the user

[lidel]

I've been looking at ways we can optimize thin window.ipfs.enable and no matter how thin we make it, it still feels wasteful to inject content script on every website and iframe.

We could make things much more efficient if we.. did not inject any script payload at all. What if websites could explicitly ask for window.ipfs, and ipfs-companion would inject it only when "invited"?

Are there any downsides to this approach?

Quick idea dump: such hint needs to happend before entire HTML is loaded, so it could be:

• presence of HTTP header in response
  • detection via webRequest.onHeadersReceived – 👍 standard WebExtension API present in Firefox and Chromium
  • requires configuration server-side, which may be an issue for people using someone's else server / public gateway outside of their control
  • open question: should we reuse existing X-Ipfs-Path as a hint, or add an explicit X-Ipfs-Api-Enable: true
    • Possible solution: X-Ipfs-Path is already sent by public gateways so it could be used as equal to X-Ipfs-Api-Enable: true (if explicit X-Ipfs-Api-Enable is not specified). This way pages loaded from public gateway would automatically get window.ipfs, but website owner would still have a way to opt-out via X-Ipfs-Api-Enable: false
• <meta> tag in HTML payload
  • detection possible via webRequest.filterResponseData – 💢 only in Firefox
• some URL-based hint
  • detection by checking URL for #x-ipfs-companion-enable-api or ?x-ipfs-companion-enable-api
  • works everywhere, does not require changes server-side, easy to opt-in but 💢 produces ugly URL

[Kubuxu]

Are there any downsides to this approach?

IMO headers are too strict. There might be cases where a user/developer is not able to control headers returned by hosting provider.

Another example might be userscripts (I know it is quite dev focused) but I think there are cases where someone might want to write a userscript using window.ipfs on third-party website.

[lidel]

Another idea:
What if we introduce a simple handshake does not require any changes on the server?

Something along these lines:

1. Website makes explicit fetch (HTTP HEAD request) for some predefined path, eg. /.well-known/ipfs/companion/window.ipfs
2. If Companion is present, it can detect such request and
   • use it as a hint to inject window.ipfs on the page that sent it via blocking executeScript call
   • redirect it to /.well-known/ipfs/companion/window.ipfs/ready to provide a confirmation everything is ready
3. If there is no companion, or window.ipfs is disabled, there will be no redirect

const handshake = await fetch('/.well-known/ipfs/companion/window.ipfs')
if (handshake.url.endsWith('window.ipfs/ready') && window.ipfs) {
  // companion with window.ipfs
  const ipfs = await window.ipfs.enable({commands: ['add','cat']})
} else {
  // no companion / window.ipfs
}

This comes with some interesting characteristics:

• improved performance, no injection by default, one universal way to "invite window.ipfs"
• works with static HTML+JS and userscripts (keeps URLs clean, no need to customize HTTP headers)
• good UX: developer decides when handshake happens (just like they can decide when window.ipfs.enable is called)
• easy to introduce into existing libraries: just update window.ipfs-fallback and everything continues to work
• use of fetch removes surface for synchronous code, solving issue of early injection in Chrome (window.ipfs injected after page loaded #362 (comment))

Downsides:

• introduces additional HEAD request (but we could make it to localhost to remove network overhead)

[lidel]

PSA: to remove overhead introduced by IPFS Companion the plan for the end of Q4 is to expose window.ipfs only when any of below hints is true:

• Origin is localhost (to make it easier for developers)
• X-Ipfs-Path header is present (indicating response comes from IPFS Gateway)
• DNSLink exists (we do async lookup anyway)
• URL is a valid content path (/ipfs/ or /ipns/ or CID in subdomain)

This will not introduce any new overhead (reusing checks we already do for other things) and will decrease resource usage on non-IPFS websites.

It will also act as a small incentive to enable DNSLink and/or self-host gateway.

[lidel]

The window.ipfs experiment was disabled since last year (see #777 for rationale why).

It did help us understand challenges and risks, and identified the need for more robust API for the web. We are officially putting window.ipfs experiment (in the form that was disabled in #777) in the Icebox and will be looking at other ways for node sharing and deduplication on the web in ipfs/in-web-browsers#158.

If we revisit this approach, it will be a more robust API tailored for use on the web,
and most likely happen after migration to Manifest V3 (#666).