RuntimeTypeInspector.js: Add RTI build target + Examples integration

[kungfooman]

Fixes #2529

I'm excited to introduce RuntimeTypeInspector.js, which I believe will benefit PlayCanvas as a whole and our debugging workflows in specific. RTI is designed to type-check the arguments of every function and method. It detects type errors while the engine is running in a way that goes beyond the capabilities of TypeScript, as demonstrated in this video (there is another one on the website debugging the Microsoft Holometric Video PC project).

During the development of RTI it already helped me to open issues or make PR's for various issues (e.g. NaN bugs).

I believe that RTI will help improve productivity and efficiency for both engine developers and users, by catching errors early on and reporting them in detail.

Runtime Type Inspector is implemented using Babel AST traversal with custom JSDoc parsing (powered by TypeScript itself for highest compatibility), customized to PlayCanvas requirements. That means for example supporting polymorphing this types.

TLDR: It adds type assertions for runtime type checking, for example:

Being able to set breakpoints exactly where the errors are happening for the first time helps to track down problems that are otherwise difficult to pinpoint (and saves a lot of time 😅).

Try for yourself: http://pc.runtimetypeinspector.org/

Just make sure to pick the right JS context - the exampleIframe:

I'm working on this for quite some time by now, a mix of fun and frustration - rewriting code on the AST level first seemed tricky, but given enough time, it was a rather straightforward way to automate type validations. I'm excited to read feedback and suggestions. 🙏

I confirm I have read the contributing guidelines and signed the Contributor License Agreement.

[Maksims]

This is pretty cool!
A couple of questions:

1. Is there a way to suppress reporting more than N errors if they are originating from the same place? Spam of errors on every application update - is not much of use.
2. Is there a way to improve the stack within console log reporting so it points to the right place of the error? This will allow easier debugging with Dev Tools, but also provide a deep linking from Launcher to IDE.

[kungfooman]

Hey @Maksims, thank you for the feedback!

1. Is there a way to suppress reporting more than N errors if they are originating from the same place? Spam of errors on every application update - is not much of use.

Yep, for this there is this button:

Currently it supports a "warn once" or "spam all the time" mode, but do you mean even more fine-grained control with N + source?

(I just uploaded to latest HEAD + fixed a issue with the spam-button-state, now it saves/restores properly from localStorage)

2. Is there a way to improve the stack within console log reporting so it points to the right place of the error? This will allow easier debugging with Dev Tools, but also provide a deep linking from Launcher to IDE.

It's a console.error and if you click on it, the stack shows up:

Then you can click on the file location under assertType and you are at the "error source" and can just put a breakpoint:

So currently it takes two clicks to get to the place of the type error, but I'm not sure what you mean with deep linking from Launcher to IDE, could you elaborate a bit on that?

Thank you again!

[Maksims]

Yep, for this there is this button:

It did not work, unfortunately. I tried it, made no difference. Still, had errors spammed:

It's a console.error and if you click on it, the stack shows up:

Ah, I did open dev tools after the errors were reported, it does not contain stack in such place, and all errors "originated" from the same place.
Is there a way to preserve their original stack? E.g. if the exceptions happen in the user script, but it reports first playcanvas.rti.js:504, then it is less clear where the error happened. This is actually an issue already with the current engine, where some errors like providing wrong arguments, are reported from within the engine (that's where they happen), but it is not straight away apparent to the user that it was due to passing wrong arguments. Especially when these arguments are not directly applied, but this is slightly unrelated.
It just highlights an issue: if an error is reported not from the user script, then they first think it is not their doing. So if an error happened somewhere within the user script or engine, the stack should specify errors from there, not from one single place.

Editor can try their best to filter through stack lines, and figure out if it can provide a deep link to a specific script in IDE, as it is really useful.

[kungfooman]

It did not work, unfortunately. I tried it, made no difference. Still, had errors spammed:

Hm, maybe the intention of the button isn't clear, when "Spam type reports" is checked it means that every type error is calling console.error (and if it is unchecked, the type errors are only logged once). If that doesn't work, it is probably because I updated the file and then it needs a Ctrl+R with DevTools open to clear the cache 🤔

Thank you for pointing out the stack trace issue when DevTools isn't open from the beginning - I wasn't aware of that, so I will save the stack traces aswell for better inspection (would be nice to see in UI too).

Editor can try their best to filter through stack lines, and figure out if it can provide a deep link to a specific script in IDE, as it is really useful.

Right, I made a little experiment what kind of information we can get out of the stack trace for Editor integration:

const info = () => {
  try {
    throw new Error(); // Dummy exception to retrieve stack trace
  } catch(e) {
    const {stack} = e;
    if (stack.toLowerCase().includes('script')) {
      const lines = stack.split('\n');
      const i = lines.findIndex(_ => _.includes('ScriptComponent'));
      let scriptLine = lines[i - 1];
      scriptLine = scriptLine.replace(/at/, '').trim().split(' ')[0];
      console.warn('Type error in script: ', scriptLine);
    }
  }
}
info(), false; // false so it doesn't breakpoint here

This is "breakpoint code" we can enter here:

(clicking on the function source)

Then you have integrated the "breakpoint code" into the code flow and now you can play around with stack traces:

Thinking about the editor, we could just add "quick links" to the actual scripts now. Thank you for the suggestion!

Edit: I think I finally understood it: Instead of "spam" and "once", it should have an option "never", so it doesn't spam anything in the console (I will work on that).

Edit 2: Added the type error reporting mode never now:

(this keeps DevTools completely "clean", only reporting in the UI)

[kungfooman]

I'm finally coming to a resolution here, it just feels so spammy for many examples still, even tho they run fine (simply based on wrong types in the engine). My main goal is "0 errors" - so developers can start editing examples and only see type errors they introduced themselves during their coding session.

If anyone has time, I would love to have these issues fixed:

• http://pc.runtimetypeinspector.org/#/graphics/paint-mesh - scopeId seems to be a left over and has to be removed?
• http://pc.runtimetypeinspector.org/#/user-interface/button-sprite - we are setting length of number arrays and move them around and calculating all kinds of stuff with them (leading to vectors filled with undefined) - useless calculations, so there is potential for a speedup aswell via small refactor
• http://pc.runtimetypeinspector.org/#/graphics/lights-baked-a-o - filling of matrices with 'NaN' values
• significantly reduce error counts, especially the ones that occur every frame

Any volunteers? 😅

[LeXXik]

A good cause 👍 I might tackle some of those next week.