Better Support for Source Breakpoints when output is in different folders

[MeirionHughes]

• VSCode Version: 1.1.1
• OS Version: Windows 10

Steps to Reproduce:

Given:

./dist/foo.js
./dist/foo.map

./spec/foo.spec.js
./spec/foo.spec.map

./src/foo.ts
./src/foo.spec.ts

1. In node launch config...
2. set "outDir": "${workspaceRoot}/dist"
3. set breakpoint in foo.ts
4. set breakpoint in foo.spec.ts
5. Run f5 (node debug)

then:
break-point will resolve for foo.ts
break-point will not resolve for foo.spec.ts

use-case:

I'm outputting source code to both a "dist" and "spec" folder; based on whether the file is *.spec.ts or not. As a result I have two locations for source mappings. In order to be able to assign breakpoints in both the .ts and .spec.ts files I must specify an outDir for VS Code's launch.json. Unfortunately, it does not support multiple directories so I can only pick one at the moment.

[MeirionHughes]

actually, could this be globbed please?

If my TS file is in a sub-directory, it doesn't seem to be able to find the breakpoints either.

[shoerob]

+1

I have source maps in some modules in the node_modules folder, and VSCode is unable to find them. If I place a breakpoint in a .ts file in a node_modules module, the vscode-node-debug debugger is unable to automatically resolve the source map location. However, if I instead place the breakpoint in the corresponding .js file, the source map resolves based on the sourceMappingURL, and the breakpoint is hit in the .ts file. This is a somewhat counterintuitive workaround. Being able to specify multiple outDirs looks like it might fix the problem.

[MeirionHughes]

I've had to solve this by compiling ts->js/map in the same folder, you can then entirely hide the js/map files that have a corresponding ts file next to them.

.vscode/settings.json:

{
  "files.exclude": {
    "**/*.js": {"when": "$(basename).ts"},
   "**/*.map": {"when": "$(basename).map"}
  }
}

This seems like the only way to reliably have VSCode map from js to ts, when execution moves over multiple folders, and make your project tree look sane.

[kineticfaction]

I think this is related to a similar problem I am having.

My projects file structure is as follows:

project

• build
  • routes
    • nestedFile.js
  • app.js
• maps
  • routes
    • nestedFile.js.map
  • app.js.map
• source
  • routes
    • nestedFile.ts
  • app.ts

With my setup I can add breakpoints to the app.ts and everything works as expected however if i add a breakpoint to nestedFile.ts get the dreaded "Breakpoint ignored because..." error.

I'm assuming that until outDir can take an array of directories as a value this is going to be a problem and the only realistic work around for larger project is as @MeirionHughes suggests?

[MeirionHughes]

I've updated the issue so its a bit more clear.

[kineticfaction]

I think that while being able to pass outDir an Array is a solution, its probably not the best solution.

Glad you changed the title of the issue. :)

[weinand]

@kineticfaction your structure should work (and I even have an automatic test for exactly that case). Set outDir to ${workspaceRoot}/build and make sure that the source maps are correct. The source maps don't need to be under outDir.

[weinand]

I agree, we need a way to specify more than one outDir in the launch.json or use a different mechanism, e.g. scanning the whole project structure upfront to locate all source that is referenced by the generated JavaScript (but this can be expensive for large projects).
Both options would solve the issue described in the initial comment.

However, in the discussions/comments that are following I see some speculation and misconceptions about how things work. So here are some clarifications:

Source mapping always involves two directions: mapping source to generated code and mapping generated code back to source. JavaScript source maps only cover the second case: "mapping generated code back to source". The other direction must be either derived from the first direction (exhaustive search), or by explicitly configuring it in the launch.json.

Let's start with how the "mapping source to generated code" is configured with the outDir attribute in the launch.json:

The outDir is only used to map a breakpoint in source to the generated file (because there is no other direct reference in the source that would help VS Code to find the generated code where the real breakpoint has to be registered). If setting a breakpoint in source fails in an active debug session (breakpoint turns into a grey donut), this indicates that VS Code cannot map the source breakpoint to the corresponding line in the generated code. Typically you can fix this by supplying the outDir and by making sure that the folder structure in source matches the structure in the generated code. If your build process generates a different structure for the generated code than the source structure, setting breakpoints will most likely fail.

Now let's look into the other direction "mapping generated code back to source":

For this the VS Code debugger needs a way to find the originating source if a breakpoint has been hit in some generated JavaScript file. By convention there is a URL at the end of the generated code file that points to the source map (*.js.map) or contains the source map inlined as a data url:

console.log("end of code"); 
//# sourceMappingURL=./app.js.map

The URL can either point to a location on disk (an absolute or relative file URL), or it can be an http URL that points to some remote file. This means that VS Code makes no assumption about where the source map is located but just follows the URL in the generated code (so the statement from some comment above "VS Code fails to match up map/js in sub-folders" is not correct). Just make sure that the URL is correct and VS Code will be able to find the source map. I recommend to either keep the source maps side by side with the generated code which results in a very simple (relative) URL or to use inlined source maps.

With inlined source maps the URL is a data URL that contains the map. This is less error prone, but since the data URL uses base64 encoding, it is more difficult to see what's in the source map.

The next step in "mapping generated code back to source" is in the source map itself. Here the entries in the sources array should point to the source files. If they are absolute paths it is easy to verify that they are valid. However it is good practice to keep the entries short by making them relative. But this makes verification a bit more difficult because you have to understand to what reference point they are relative. The source map spec says: "If the sources are not absolute URLs after prepending of the “sourceRoot”, the sources are resolved relative to the SourceMap (like resolving script src in a html document)."

So in @kineticfaction's structure from above the entries in the sources array of the source map would look like this:

{
   "file": "nestedFile.js",
   "sources": [
      "../../maps/source/routes/nestedFile.ts"
   ]
}

and

{
   "file": "app.js",
   "sources": [
      "../maps/source/app.ts"
   ]
}

Or if a common prefix of the paths is extracted into a separate sourceRoot attribute:

{ 
  "file": "nestedFile.js",
   "sourceRoot": "../../maps/source",
   "sources": [
      "routes/nestedFile.ts"
   ]
}

and

{
   "file": "app.js",
   "sourceRoot": "../maps/source",
   "sources": [
      "app.ts"
   ]
}

[MeirionHughes]

Typically you can fix this by supplying the 'outDir' and by making sure that the folder structure in source matches the structure in the generated code.

Doesn't appear to work properly for me when I have subfolders. I have some code in ./src/rules/... that don't match up with ./dist/rules/... when I use outDir: "./dist/"

[weinand]

@MeirionHughes you have to set outDir to the generated code, not the source.

[MeirionHughes]

sorry yes... typo. it is set to dist.

In my case it definitely matches up the root folder though: (./src to ./dist)

[weinand]

@MeirionHughes and it must be an absolute path, e.g. ${workspaceRoot}/dist

[MeirionHughes]

Yes sorry; its exactly that. :P

[MeirionHughes]

ahh I may know why... I think my sourceRoot is wrong in the output. the relative path from the sub folders is wrong. I'm setting it like:

.pipe(sourcemap.write('.', { sourceRoot: '../source' }))

[weinand]

Can you please add "trace": "sm" to your launch config and then set a breakpoint. In the debug console you should see a trace of what's going on. That could help investigating this.

[MeirionHughes]

Sorry @weinand, you are correct with the same folder structure plus correct sourceRoot; it does work.
Seems there is an issue with gulp-typescript or gulp-sourcemap with regard to setting the relative path see( ivogabe/gulp-typescript#395 ).

So we're left with the original issue (support for multiple outDir folders).

[weinand]

@MeirionHughes great to hear that sourcemaps works for you for the one-outDir case.

I'm planning to add globbing support for outDir soon.

[kineticfaction]

Thanks Andre I'll take a look this evening once i get out of the office

[kineticfaction]

@weinand Yep works perfectly as expected.

[weinand]

@kineticfaction thanks a lot for verifying that your setup works.

[xuld]

How about debugging codes generated by ts-node?

I'm using mocha to run test.ts:

mocha --require ts-node/register test.ts --debug

ts-node will compile test.ts to test.js which locates in a temp folder. The folder structure in source can never match the structure in the generated code.

I can't debug for test.ts if I set the sourceMap option enabled.