asExternalUri doesn't work #1936

orshefi · 2020-08-02T08:52:16Z

Hi Guys,

I had written a new extension which renders Graphs from matplotlib into vscode webview component, it works great on vscode desktop, but I encounter problems when trying to work with it in code-server.

I cannot state what exactly the problem is but here is the workflow:

my vscode extension starts and listens to requests for plot viewing on port 11111 (POST request)
matplotlib custom backend sends a request to the extension with the url to open in webview
I use vscode.env.asExternalUri tunneling in order to open the url in a webview and embed the generated address in an Iframe.
Usually by now the webview with the webpage would have been loaded, but instead I get and error openning localhost refuse to connect

Any ideas what went wrong?

Web Browser: Chrome
Local OS: Ubuntu
Remote OS: MacOS
Remote Architecture: x86_64
code-server --version: v3.4.1

The text was updated successfully, but these errors were encountered:

code-asher · 2020-08-03T15:28:02Z

I'm not too familiar with asExternalUri but this might mean there's something missing in our implementation. We'll need to dig further.

orshefi · 2020-08-04T08:36:07Z

Thanks for the response, I actually solved it from another direction.

Instead of using iframe to the rendered graph url on the host, I took its html and embedded it into the webview.
I also created a translator from the WebSocket that matplotlib webagg backend creates to vscode Iframe messaging infrastructure.

It works ok, you can close this issue.

I believe soon I'll release this extension to be open source and then I'll need your help to get it into your extensions server

code-asher · 2020-08-04T16:24:52Z

Awesome, that sounds like a really good workaround.

I'm thinking we should leave the issue open until we fix asExternalUri since it would be ideal to have parity with local VS Code.

stale · 2021-10-26T22:19:11Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no activity occurs in the next 5 days.

jsjoeio · 2022-06-30T16:18:19Z

The Tabnine team is still facing this so we need to dig into it.

misskniss · 2022-07-26T17:48:51Z

If the work around is not working @code-asher @jsjoeio @Emyrk what are options for addressing this?

jsjoeio · 2022-08-01T19:39:51Z

I'm guessing @code-asher and I will need to add a patch to fix this then.

code-asher · 2022-08-02T18:41:57Z

To fully fix for the Tabnine case we will need:

Implement asExternalUri.
Use the sub-domain proxy instead of path-based proxy when available. Tabnine will only work with sub-domains since their application does not use relative paths and it does not seem they are able to fix that or provide a workaround like allowing a base URL. Users will need to set up wildcard certificates and DNS (or at the very least one for the Tabnine port).
Allow external proxies (like Coder's dev URLs) so it can be set up to work in environments that have their own sub-domain proxy setup and cannot delegate that role to code-server.

ksylvan · 2022-09-11T21:01:14Z

Any updates on this issue? I’m still running into problems using TabNine on VS Code Web.

jsjoeio · 2022-09-12T18:37:18Z

Chatting with their team about this. Stay tuned!

Sominemo · 2022-09-13T06:38:45Z

Dart & Flutter extension for VS Code uses asExternalUri too, and its DevTools iframes don't work without manual port forwarding, because it tries to connect to localhost. As far as I understand, it's related to this issue.

jsjoeio · 2022-09-14T17:16:18Z

Thanks for letting us know @Sominemo! That'll be helpful when we go to fix this. We may want to write our own extension that uses asExternalUri and add an e2e test to make sure it works (when we fix this).

jsjoeio · 2022-10-04T17:56:08Z

Notes

I have started looking into this and will share my notes here and update this comment as I go.

CONCLUSION: I successfully reproduced the bug.

Investigation: does `asExternalUri` work for a basic callback URL? ✅

Here are the steps I'm following to check:

create a new extension: npx yo code
open code-server (4.7.0) locally (macOS + Brave) and open extension
use vscode.env.asExternalUri in extension (see code below)
run extension (F5) or click "Run extension" in Run and Debug window
run command and test

      const callableUri = await vscode.env.asExternalUri(
        vscode.Uri.parse(vscode.env.uriScheme + "://test")
      );

      vscode.window.showInformationMessage(
        `your url: ${callableUri.toString()}`
      );

Expected
It shows callback URL in information notification and that URL is accessible in my browser.

Actual
It shows callback URL in information notification and that URL is accessible in my browser.

Investigation: does `asExternalUri` work for a URL like `localhost:8000`? ❌

Similar steps as above but with this code:

      const callableUri = await vscode.env.asExternalUri(
        vscode.Uri.parse(`http://localhost:8000)
      );

      vscode.window.showInformationMessage(
        `your url: ${callableUri.toString()}`
      );

Expected

It behaves like Codespaces and transforms the URL to be behind the a proxy such as https://jsjoeio-coder-code-server-rpvvw55xp7f4x9-8000.githubpreview.dev/. More info needed here on how this should behave.

Actual

It does not transform URL and returns http://localhost:8000.

jsjoeio · 2022-10-04T22:32:45Z

I've identified the Root Cause as well and have an idea for fixing this.

Root Cause

Based on my findings, the call stack looks like this:

extension calls asExternalUri using vscode.env.asExternalUri
that calls asExternalUri here
that calls $asExternalUri here
calls openerService.resolveExternalUri here

This last function -- resolveExternalUri -- tries to resolve this external uri using one the _resolvers (property on class). The problem is...there are no resolvers in the browser (at least in code-server, where i'm running with yarn watch).

"Okay...so shouldn't it throw this error?"

You would think so but if you look at the link in 3 in my call stack list, you'll notice the code isn't wrapped in a try/catch block. Therefore, that error never bubbles up.

"is that a bug?"

I would say so but I might check with @code-asher. Raised upstream: microsoft/vscode#162770

There may be more to this root cause since I'm not sure why there wouldn't be a resolver for this external URI in the first place. I know it works in Codespaces but I can't figure out how.

Solution

I'd like to discuss with @code-asher first but I did find this. When I add resolveExternalUri here called like so:

		resolveExternalUri: async (uri: URI) => {
			const v = await Promise.resolve(URI.parse("http://localhost:8080/proxy/8000"))
			return v
		},

then my extension uses it!

If this is the correct place to patch, then next steps would be to agree upon how external uris should be resolved. One approach could be to use code-server's built-in proxy and do something like localhost:8000 -> <base-url>/proxy/8000.

jsjoeio · 2022-10-06T15:14:27Z

I chatted with @code-asher offline and we came up with a fix. I'm going to work on it today and hopefully push up a draft PR soon. Keep an eye out.

jsjoeio · 2022-10-07T22:20:04Z

Good news - PR is pretty much good to go. We should be able to wrap it up Monday assuming all goes well.

ksylvan · 2022-10-09T22:31:50Z

Awesome. Thank you!

ksylvan · 2022-10-11T22:30:09Z

Good news - PR is pretty much good to go. We should be able to wrap it up Monday assuming all goes well.

Would you like me to try testing this?

jsjoeio · 2022-10-13T16:44:06Z

@ksylvan yes please!

P.S. I should have time to clean up my PR today and merge 🤞🏼

ksylvan · 2022-10-13T17:32:33Z

What's the easiest way for me to test this?

jsjoeio · 2022-10-13T18:09:09Z

Good question! Easiest way that comes to mind:

yarn init -y
yarn add @coder/code-server-pr@4.7.1-5624-6ae9dec77528bc9af1b1de503d56ea5318bab99b
yarn code-server --auth none --port 3000

That'll load code-server with the changes. Then you could try using any basic extension that uses asExternalUri and it should work. I think Tabnine will need changes upstream after our PR is merged.

ksylvan · 2022-10-13T18:23:25Z

That'll load code-server with the changes. Then you could try using any basic extension that uses asExternalUri and it should work. I think Tabnine will need changes upstream after our PR is merged.

$ yarn add --verbose @coder/code-server-pr@4.7.1-5624-6ae9dec77528bc9af1b1de503d56ea5318bab99b
yarn add v1.22.19
[...]
verbose 2.436532975 Request "https://registry.yarnpkg.com/balanced-match" finished with status code 200.
verbose 2.458458046 Request "https://registry.yarnpkg.com/type-check" finished with status code 200.
verbose 2.494140741 Request "https://registry.yarnpkg.com/esutils" finished with status code 200.
[2/4] Fetching packages...
error @coder/code-server-pr@4.7.1-5624-6ae9dec77528bc9af1b1de503d56ea5318bab99b: The engine "node" is incompatible with this module. Expected version "16". Got "12.22.12"
verbose 2.83002365 Error: Found incompatible module.
    at MessageError.ExtendableBuiltin (/home/kayvan/yarn/node_modules/yarn/lib/cli.js:721:66)
    at new MessageError (/home/kayvan/yarn/node_modules/yarn/lib/cli.js:750:123)
    at checkOne (/home/kayvan/yarn/node_modules/yarn/lib/cli.js:47864:11)
    at Object.check (/home/kayvan/yarn/node_modules/yarn/lib/cli.js:47883:5)
    at /home/kayvan/yarn/node_modules/yarn/lib/cli.js:7352:73
    at Generator.next (<anonymous>)
    at step (/home/kayvan/yarn/node_modules/yarn/lib/cli.js:310:30)
    at /home/kayvan/yarn/node_modules/yarn/lib/cli.js:321:13
error Found incompatible module.

ksylvan · 2022-10-13T18:41:14Z

Looks like my Debian instance wasn't updated to NodeJS16. Fixed that and now seeing this:

$ yarn add @coder/code-server-pr@4.7.1-5624-6ae9dec77528bc9af1b1de503d56ea5318bab99b
yarn add v1.22.19
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning "@coder/code-server-pr > @coder/[email protected]" has unmet peer dependency "@google-cloud/logging@^9.2.1".
[4/4] Building fresh packages...
[2/2] ⠂ @coder/code-server-pr
error /home/kayvan/code-server/node_modules/@coder/code-server-pr: Command failed.
Exit code: 1

And then when I run it, I see:

yarn run v1.22.19
$ /home/kayvan/code-server/node_modules/.bin/code-server --auth none --port 3000
[2022-10-13T19:34:54.902Z] info  code-server 4.7.1-5624-6ae9dec77528bc9af1b1de503d56ea5318bab99b 6ae9dec77528bc9af1b1de503d56ea5318bab99b
[2022-10-13T19:34:54.906Z] info  Using user-data-dir ~/.local/share/code-server
[2022-10-13T19:34:54.919Z] info  Using config file ~/.config/code-server/config.yaml
[2022-10-13T19:34:54.920Z] info  HTTP server listening on http://100.79.43.44:3000/
[2022-10-13T19:34:54.921Z] info    - Authentication is disabled
[2022-10-13T19:34:54.921Z] info    - Not serving HTTPS
Loading "minimist" failed
Error: Cannot find module 'minimist'
Require stack:
- /home/kayvan/code-server/node_modules/@coder/code-server-pr/lib/vscode/out/bootstrap-amd.js
- /home/kayvan/code-server/node_modules/@coder/code-server-pr/out/node/util.js
- /home/kayvan/code-server/node_modules/@coder/code-server-pr/out/node/cli.js
- /home/kayvan/code-server/node_modules/@coder/code-server-pr/out/node/entry.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:956:15)
    at Function.Module._load (node:internal/modules/cjs/loader:804:27)

jsjoeio · 2022-10-13T19:57:46Z

Hmmm...I'm not sure why it's not installing on the modules. Here is one solution: #5530 (comment)

You could also try with npm instead. Sorry for the trouble!

ksylvan · 2022-10-26T01:31:47Z

I see the new code with the proxy server auto-forwarding is in code-server, do you know what needs to happen with tabnine extension to use it?

jsjoeio · 2022-10-26T15:26:51Z

Hmm...I'd probably have to dig in but ideally they use asExternalUri to resolve the uri for their server. Something like:

// pseudo code
const serverURL = await vscode.asExternalUri("http://localhost:3000")

code-asher added the needs-investigation label Aug 3, 2020

code-asher changed the title ~~Opening localhost iframe in web-view~~ asExternalUri doesn't work Aug 4, 2020

code-asher added the bug Something isn't working label Aug 4, 2020

code-asher mentioned this issue Aug 7, 2020

How to access host:port service running inside code-server via WebView #1949

Closed

nhooyr removed the needs-investigation label Dec 7, 2020

jsjoeio added this to the Backlog milestone Apr 29, 2021

stale bot added the stale label Oct 26, 2021

stale bot closed this as completed Nov 2, 2021

jsjoeio reopened this Jun 30, 2022

stale bot removed the stale label Jun 30, 2022

jsjoeio added the high-priority This issue needs to be resolved ASAP label Jun 30, 2022

jsjoeio modified the milestones: Backlog, July 2022 Jun 30, 2022

misskniss assigned code-asher Jul 26, 2022

jsjoeio modified the milestones: v4.5.2, August 2022 Aug 15, 2022

jsjoeio modified the milestones: August 2022, On Deck - High Priority Aug 24, 2022

jsjoeio modified the milestones: On Deck - High Priority, September 2022 Sep 14, 2022

jsjoeio modified the milestones: September 2022, October 2022 Sep 30, 2022

jsjoeio mentioned this issue Oct 4, 2022

Can't login the github and tabnine #4762

Closed

jsjoeio assigned jsjoeio and unassigned code-asher Oct 4, 2022

jsjoeio mentioned this issue Oct 6, 2022

fix: add handle for resolveExternalUri #5624

Merged

4 tasks

jsjoeio closed this as completed in #5624 Oct 14, 2022

jsjoeio modified the milestones: October 2022, 4.8.0 Oct 24, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

asExternalUri doesn't work #1936

asExternalUri doesn't work #1936

orshefi commented Aug 2, 2020

code-asher commented Aug 3, 2020 •

edited

Loading

orshefi commented Aug 4, 2020

code-asher commented Aug 4, 2020

stale bot commented Oct 26, 2021

jsjoeio commented Jun 30, 2022

misskniss commented Jul 26, 2022

jsjoeio commented Aug 1, 2022

code-asher commented Aug 2, 2022 •

edited

Loading

ksylvan commented Sep 11, 2022

jsjoeio commented Sep 12, 2022

Sominemo commented Sep 13, 2022

jsjoeio commented Sep 14, 2022

jsjoeio commented Oct 4, 2022 •

edited

Loading

jsjoeio commented Oct 4, 2022 •

edited

Loading

jsjoeio commented Oct 6, 2022

jsjoeio commented Oct 7, 2022

ksylvan commented Oct 9, 2022

ksylvan commented Oct 11, 2022

jsjoeio commented Oct 13, 2022

ksylvan commented Oct 13, 2022

jsjoeio commented Oct 13, 2022

ksylvan commented Oct 13, 2022 •

edited

Loading

ksylvan commented Oct 13, 2022 •

edited

Loading

jsjoeio commented Oct 13, 2022

ksylvan commented Oct 26, 2022

jsjoeio commented Oct 26, 2022

asExternalUri doesn't work #1936

asExternalUri doesn't work #1936

Comments

orshefi commented Aug 2, 2020

code-asher commented Aug 3, 2020 • edited Loading

orshefi commented Aug 4, 2020

code-asher commented Aug 4, 2020

stale bot commented Oct 26, 2021

jsjoeio commented Jun 30, 2022

misskniss commented Jul 26, 2022

jsjoeio commented Aug 1, 2022

code-asher commented Aug 2, 2022 • edited Loading

ksylvan commented Sep 11, 2022

jsjoeio commented Sep 12, 2022

Sominemo commented Sep 13, 2022

jsjoeio commented Sep 14, 2022

jsjoeio commented Oct 4, 2022 • edited Loading

Notes

Investigation: does asExternalUri work for a basic callback URL? ✅

Investigation: does asExternalUri work for a URL like localhost:8000? ❌

jsjoeio commented Oct 4, 2022 • edited Loading

Root Cause

Solution

jsjoeio commented Oct 6, 2022

jsjoeio commented Oct 7, 2022

ksylvan commented Oct 9, 2022

ksylvan commented Oct 11, 2022

jsjoeio commented Oct 13, 2022

ksylvan commented Oct 13, 2022

jsjoeio commented Oct 13, 2022

ksylvan commented Oct 13, 2022 • edited Loading

ksylvan commented Oct 13, 2022 • edited Loading

jsjoeio commented Oct 13, 2022

ksylvan commented Oct 26, 2022

jsjoeio commented Oct 26, 2022

code-asher commented Aug 3, 2020 •

edited

Loading

code-asher commented Aug 2, 2022 •

edited

Loading

jsjoeio commented Oct 4, 2022 •

edited

Loading

Investigation: does `asExternalUri` work for a basic callback URL? ✅

Investigation: does `asExternalUri` work for a URL like `localhost:8000`? ❌

jsjoeio commented Oct 4, 2022 •

edited

Loading

ksylvan commented Oct 13, 2022 •

edited

Loading

ksylvan commented Oct 13, 2022 •

edited

Loading