-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Inspecting Node.js with Chrome DevTools #2546
Comments
/cc @trevnorris, @bmeck |
/cc @nodejs/tracing |
I did some related work a while ago: https://github.com/thlorenz/debugium Implementing the protocol is the easy part. However trying to reuse code from chromium that communicates with v8 turned out to be non-trivial. For anyone working on this https://github.com/thlorenz/chromium-remote-debugging-proxy may come in useful as it allows watching the messages sent between chrome DevTools and the debugged instance which helps in understanding the protocol used. |
To add a few items… This project brings all of the DevTools's JS features to Node. That includes:
Moving forward with this would make the current node-inspector codebase somewhat redundant, but we'd like to work with @bajtos, @3y3 and friends on making sure the experience of launching a debugging/profiling session with these tools is smooth and enjoyable.
This ticket is the materialization of that work. ;) @yury-s is an engineer on Chrome and owns the JS debugging tools and how they interface with V8. |
@paulirish Awesome! Happy this is finally happening :) |
Does Will we have to integrate the same type of code into our JS, along with our C++, like in if (DEBUG_IS_ACTIVE) {
%DebugPromiseEvent({ promise: promise, status: status, value: value });
} Having this w/o LTS support reduces its utility in node. As such it will become a maintenance burden across 4 or 5 active branches. Is it even possible that this API is integrated in a V8 release that's 1-2 years old? Also in regards to LTS support, I assume that there isn't indefinite support of Chrome to debugger API support. Or am I wrong? |
It can be bound to any interface that we want, I don't see any issues with that (except security). Chrome already accepts both host and port as values of --remote-debugging-targets and the host may well be remote one. It makes sense to differentiate between transport and the debugger backend implementing the remote debugging protocol. Same debugger implementation can be used with different types of connections similar to how it works in Chrome at the moment where various transports are used to transfer DevTools traffic depending on the inspected target:
If Node uses alternative implementation of Promises and wants to leverage debugging capabilities provided by v8-inspector then yes, you will need to provide corresponding instrumentation that way or another. v8-inspector is going to have C++ API so you'll need at least one binding to notify the debugger. The call will go through the v8-inspector C++ API not through some internal v8 API.
I'm afraid not, as it would require making v8-inspector work with such old releases of v8. This is a non-goal for the project. As I mentioned, the idea is to have particular version of v8-inspector compatible with particular version of v8 API. If v8 is updated, v8-inspector will likely need to be updated as well.
At the moment, there is no official v8 debugger API which would be well-supported. There is a set of C++ methods in v8-debug.h as well as debug context which provides access to the debugger internals. This is one of the issues we'd like to address with v8-inspector. It should become a recommended way to debug v8. Eventually we'd like to drop most of v8/src/debug/debug.js and prohibit using the debug context directly. Such approach would leave us the freedom of changing remote debugging protocol without committing to its backward compatibility as there will always be a version of Chrome DevTools compatible with given version of the backend. There are already two types of protocol commands: public and hidden. Public ones are those for which we commit to backward compatibility. We're hesitant on adding more public commands as it limits our development pace. In terms of LTS, similar to v8 we don't want to commit to keeping the API stable as it would be too restrictive. What we can do is to follow the same public API compatibility as v8 does (or at least declares): https://code.google.com/p/v8-wiki/wiki/Source#V8_public_API_compatibility Since Blink is going to use v8-inspector too, we'll have to keep it up to date with latest v8 API changes. Should we make any breaking changes they will be easy to detect in nightly builds and can be reverted. On our side we'll have to make sure that changes to v8-inspector API would be made in a non-breaking manner, follow the same public API deprecation policy as v8 does and the new APIs would work for Node too, not only for Chrome. Will this work for you? |
At current iteration v8-debug has experimental method enableWebkitProtocol Main problem of this experiments - compatibility:
For this reason I also worried about @trevnorris questions:
I will be happy, if @yury-s , are you want to backport async call stack from chrome to node? And some other features? I'd like to think about |
/cc @auchenberg |
@yury-s Thank you for the comprehensive response.
Our
While I understand, it's also unfortunate. Our main stable release is 6 month. Taking into account V8's rapid release cycle I will make the assumption that the same API could break before that is over. If we can't guarantee support for at least the 6 months of latest stable then I'm afraid that it won't do us any good. Beyond that, saying it would be available for the 6 months of stable but not for the 18 months of LTS would need to have serious discussion as to whether bringing it is still viable. For the time I'm ignoring the additional 12 months a release will be in maintenance. Which brings full support to 3 years. I do hope we can work out these issues, as having this available would be very helpful. |
No. See my reply above.
As Paul Irish wrote above the project will come with all debugging features available in Chromium. Async call stacks is not an exception. There is a generic API resembling the one described in this MSDN article which needs to be called from appropriate places where we'd like to capture async call stacks. I can help with adding such instrumentation to Node but we'll need some person knowledgeable better than me to identify places where the hooks should be added.
This is a good way to think about v8-inspector. Ideally, we should be able to drop v8-debug.h in favor of v8-inspector. This would make the API more explicit and allow to provide better support for it. |
How does this work with v8 at the moment? There may be 4 or 5 major v8 releases in 6 months which means there may be a lot of breaking changes. Assuming that v8-inspector is shipped as part of v8 and its API is defined in /include/v8-inspector.h (which would be ideal case for us long-term) how would that work with Node?
I hope so too. |
Once a major release branch is cut from
From what I've read, it wouldn't. It doesn't make sense for us to integrate an API that will break so quickly. It's an added disadvantage since Google is insistent that no one have older versions of Chrome installed. If we were allowed to do that then it might be possible to simply use the corresponding Chrome version with a given node release. |
I think it is very important that the debugger ships with node as a standalone (rather than connected via I think it would also be very important that all the instrumentation hooks we need for async are there - and that integration when running non js modules in node works (reasonably) well. Of course, userland hooks might also be interesting (so people can signal to the debugger things they want it to ignore, for instance - C# has this and it's very useful). Huge +1 for the idea by the way. |
This is great! Glad to see this is moving forward.
As I understand it, the debugger back-end is separate from the core via By having a runtime/protocol version we will be able to match the front-end with the back-end. Eventually this would need to use some kind of feature detection, but that's further out in the future, as I see it. |
Yeah, the only way I see this working is shipping the UI as an electron app with each release, or ensuring backwards compatibility in the protocol long enough to support the LTS releases. I think backwards compatibility for the dev tools should, in theory, be easier than V8 itself. It's only the protocol that is important here. I don't expect anyone should be touching the API outside of core, so node core doesn't need to worry about API compatibility. |
@yury-s Meant to ask, how much do you think the actual debugging API that connect client to process would change? If that itself doesn't change much, or can be fixed more easily over time, then we'd be much closer to having a workable solution. |
@benjamingr @Qard from the OP:
Doesn't it mean that one could use the latest Chrome and still be able to debug any version of Node.js using an old API ? |
@targos I'd much prefer it if it was possible to run the debugger without accessing code served from the cloud live. Node would have no control over which versions are supported and how - nor would it be able to easily edit the code (unless node starts serving it). I think it would be preferable to have a standalone app. |
Node isn't going to control what tools are used to communicate with the debugger API regardless. I'm not sure how some companies would feel about needing an internet connection to debug processes on their local network, but I'd say that sounds like a promising solution. |
Yeah, not sure how I feel about a service out on the net somewhere having
|
Would the scope include things like |
This feature could also lead to use the Firefox devtools with node thought the valence project 👍 👍 👍 |
This one is stable. For basic debugging support it is as simple as this: class FrontendChannel {
virtual void sendToFrontend(const String& message) = 0;
}
class Backend {
void connectFrontend(FrontendChannel* channel);
void disconnectFrontend();
void dispatchMessageFromFrontend(const String& message);
} See https://github.com/yury-s/v8-inspector/blob/embedded-in-io/Source/v8inspector/V8Inspector.h#L33 and https://github.com/yury-s/v8-inspector/blob/embedded-in-io/Source/core/inspector/InspectorFrontendChannel.h The messages are strings that are opaque to Node, all parsing and dispatching of the protocol messages is performed by v8-inspector itself.
At that point v8-inspector is also branched along with v8. Also we know exact revision of the front-end that is compatible with that v8-inspector and can use it for debugging this particular release of v8 (embedded in Node, Chrome or something else). At the moment v8 branching is consistent with Chrome so we should always have good front-end that is capable of debugging that v8 version. More details below.
@targos exactly. We already allow debugging Chrome 38 that can be found on many Android devices by connecting to it with Chrome 46. The way it works is for each Blink revision we have a copy of the front-end sources served from the cloud. After that we ask inspected target which front-end revision should be used for debugging and load it from the cloud into devtools window. E.g. to connect to chrome 44.0.2403.133 we use this link https://chrome-devtools-frontend.appspot.com/serve_rev/@199588/inspector.html In case of Android remote debugging we load the front-end into a browser window that has special devtools bindings providing connection to the browser on the device over USB. This API is backwards compatible so that we could load old front-ends. In case of Node we don't even have to do this since we can connect via plain WebSocket . The front-end can be loaded as a normal page and remote target url can be passed as 'ws' param: https://chrome-devtools-frontend.appspot.com/serve_rev/@199588/inspector.html?ws=localhost:9222/node This will work except for some features that require more privileges than regular web page can have, e.g. access to local file system. Loading as a privileged DevTools front-end provides better user experience. |
The front-end code can be deployed in a different manner, e.g. it could be provided as a .deb package. I just described how it is implemented in Chrome at the moment and it has been working pretty well for providing zero-conf Android debugging experience across all platforms. Being able to see Node instances as another type of inspectable target in chrome:inspect looks natural to me but this doesn't have to be the only way to debug Node. |
@jkrems :
At the moment it includes Debugger, Runtime, Profiler and HeapProfiler agents which make sense for all v8 embedders. Console could be added to the list later. On the other hand, PageAgent doesn't make much sense for Node so we'd rather keep it in Blink. If we need "reload" functionality, the method could be moved to e.g. to Runtime domain or Node can have its own domain "Node" providing functionality specific to Node.js The front-end can either support this in the core or we can have it as a third-party module. This is definitely not something for v.1 and deserves separate discussion. |
Correct. Other front-ends should be able to connect using the same transport. Note though that we don't want to commit to backward compatibility of the whole protocol. It would be up to the front-end authors to make sure that their front-end is compatible with given back-end. We only promise to provide compatible version of DevTools front-end.
Note that this version only relates to the subset of public commands in the protocol. We have been reluctant to making more commands public so the protocol version hasn't changed for a while. Majority of the commands in the protocol are non-public and they have been changing quite often. To ensure full compatibility with the protocol discovery page returns exact version (198849) of the front-end that should be used. |
I just opened a PR with initial v8_inpector support: #6792. |
Can this be closed now? |
I think so. Please reopen if I'm wrong. |
What is the current status on the effort to integrate in with vanilla v8? I'm currently building this functionality and have it minimally functional but would rather not, but I don't know how to find out about the progress being made. Thank you. |
@xaxxon use |
I'm not using node, though. On Wed, Oct 5, 2016 at 8:23 AM, Bradley Meck [email protected]
|
You might want to check the V8 issue tracker then. |
The code is in the V8 now, e.g. - On Wed, Oct 5, 2016 at 8:24 AM Colin Ihrig [email protected] wrote:
|
thank you. On Wed, Oct 5, 2016 at 9:24 AM, Eugene Ostroukhov [email protected]
|
I wonder if it is currently or would be possible to access this url programatically from the Node process? |
No, not yet. We can expose it if it is really needed. |
I'm just think it would be nice to be able to expose it so in the process can open the tab for you automatically if you so wish. However It seems that you can actually determine the URL yourself if you know the port, which you can specify with |
UUID is not exposed. On Wed, Nov 2, 2016 at 9:55 AM Matthew Spence [email protected]
|
With
Where |
I was having the same issue a few days ago and wrote a Chrome extension to solve it. Would love any feedback. Direct Chrome Web Store link: https://chrome.google.com/webstore/detail/nim-node-inspector-monito/gnhhdgbaldcilmgcpfddgdbkhjohddkj |
Objective
We’ve been thinking about providing unified debugger support across various v8 embedders. One natural approach to that is to reuse Chrome DevTools and provide its JS debugger, performance and memory profiler functionality to Node.js users. What we aim for Node is what we have for Chrome for Android. You basically go to chrome://inspect on a stable version of Chrome and it debugs your running Node instance. You might need to override / specify ports to both if you want them to be non-default. The rest is behind the scenes.
Internal design
DevTools is comprised of the two major components: Frontend providing UI and Backend that instruments inspected application. Frontend is essentially a web application that communicates to the Backend via remote debugging protocol and nothing prevents it from connecting to Node.js instead of Chrome. To support that we’ll need to extract parts of debugger and profiler functionality from Blink into a separate project that would implement DevTools remote debugging protocol. Let’s call the project v8-inspector. Node.js could then expose the protocol to the Frontend over a WebSocket connection.
Proof of concept
I’ve created a prototype demonstrating this approach:
https://github.com/yury-s/v8-inspector - implementation of DevTools protocol for debugger, performance and memory profilers. I started with forking Blink and then removed dependencies that wouldn’t make sense in Node.js. There is definitely more work to be done but I believe this is good enough as a proof of concept.
https://github.com/yury-s/io.js/tree/remote-debugger - io.js clone that uses the project decribed above and ws module to allow using DevTools Frontend for debugging/profiling.
Local Node debugging scenario with custom ports would look like this:
Longer term
Long-term solution would be to have both Blink and Node.js use the same native v8-inspector library linked against v8. DevTools Frontend development would continue in Blink. For each version of v8-inspector there is a compatible version of DevTools Frontend served from the cloud. This is how it works with Android debugging at the moment: when the user opens DevTools for Chrome running on Android, the desktop browser loads a version of DevTools UI corresponding to the version of Chrome running on Android and forwards all traffic from the device to that Frontend.
Open questions
Before moving forward with this approach and investing into Blink refactorings necessary for extracting common code into its own library we’d like to collect some feedback on whether Node.js community would be interested in that. In particular, is there an interest in integrating such debugger library into Node.js core?
The text was updated successfully, but these errors were encountered: