[RFC] Testing Browser Compatibility

[nhunzaker]

Testing browser quirks is challenging. What if we set up a page or site local to the React repository that made it easier to isolate browser differences and document known issues:

For example:
http://natehunzaker.com/react-issue-reproducer/cases/input-kitchen-sink/index.html

We could start with some of the outstanding input issues (like #7359), and work our way backward to historical decision points for adding additional rules to support specific browsers.

Speaking with @gaearon and @aweary about this. We see a couple of outstanding tasks:

• Add a page/site local to the React repository that includes test cases for reproducing specific browser quirks
• Make it easy to switch between versions of React via CDN references and local builds
• Catalog existing fixes and outstanding issues, start building test cases
• Modify React flags via query strings, or some other mechanism to quickly switch between the Fiber and and standard DOM renderer.

Eventually, this could also be a great resource for how React handles DOM manipulation, obscure DOM API concepts, and reference notes for implementors of other DOM rendering libraries.

Test cases

• IE 11 and Edge no longer prompt to remember password on controlled form #7328: IE Password fields - @nhunzaker
• Backport input fix #8575: Backport input fix - @jquense
• Change event does not fire in IE and Edge under certain circumstances #4672: Change event does not fire in IE and Edge under certain circumstances - @jquense
• OnChange randomly misses keys on IE if typing very quickly #7027: OnChange randomly misses keys on IE if typing very quickly - @jquense
• onClick firing for disabled buttons in v15.4.0 #8308: onClick firing for disabled buttons - @aweary
• IE11 block change events if disabled #4457: IE11 block change events if disabled @aweary
• Mouse enter and leave work consistently with disabled elements #5762: Mouse enter and leave work consistently with disabled elements @aweary
• Change event bubbling doesn't work with controlled components in IE #708 Change event bubbling doesn't work with controlled components in IE

[nhunzaker]

cc @jquense, you've done a lot of work in this area too. What do you think?

[aweary]

A few thoughts/questions on the high-level structure of the page(s):

• How should we group test cases? Should each case have its own page and we maintain a central directory for navigating between them, or should we keep the page count down and put all related tests on a single page?

• We'll want to include exact steps on what to do to reproduce and check the behavior we're looking to validate. It should be easy for a new contributor to look at it, follow the steps, and know if its working or not.

• We can potentially use paths to load any arbitrary version of React from unpkg on any page. For example /15.4.1/input-test would load the test with React 15.4.1. This would make it very easy + quick to test regressions.

[gaearon]

I would put everything on a single page and have a switch at the top that points to CDN versions or local build, and refreshes the part of the page below when you change them.

[bvaughn]

Make it easy to switch between versions of React via CDN references and local builds

I have a small set of in-browser tests for react-virtualized that read query param and toggle between different versions of react (by version number) and react-virtualized (either a version number pulled from CDN or "local" which runs against the local build). Very useful when checking for regressions or perf changes.

+1 to Dan's in-page switch suggestion. The most common case is probably to compare between local and latest release on NPM.

[jquense]

this sounds really helpful in general. So much of that knowledge now is either lost, or trapped in tons of jsfiddles in many different issues. I'd love if the many many browser decisions made to attend to browser quirks was better documented, ITs always tough to try and understand even the starting point for addressing DOM issues without that knowledge. I'd also add that documenting the minimal reqs for certain changes would be really helpful. Such as input components needing to work with imperative sets to value on the DOM node, as well as prop changes.

Would this also be a tool for new issues? I wonder what adding to such a page would look like? Changes accompany associated fixes? Is it possible to leverage an existing code iteration tool (codepen/jsbin/jsfiddle) for this? That might make it more helpful for the beginning bits of identifying new issues.

[aweary]

As I see it, the process would look something like:

1. User identifies new issue and shares a case reproducing it (JSFiddle or whatever)

2. We verify the issue

3. Someone fixes it and includes the changes to the test page in the PR just like they do now with unit tests

4. We now use the test page to validate the fix in our supported browsers

[aweary]

This is a long term stretch goal, but automatic deployment of the test page when a PR is opened that affects DOM code would be amazing.

[nhunzaker]

One other item, though I'd be okay with excluding it as a core part of this issue, is base line browser compatibility.

• What browsers do we target?
• Should there be an effort to strip fixes for unsupported browsers, such as IE8?
• If not, should we consider a compatibility package? I think this primarily pertains to the event plugin system, though I could be wrong.

[aweary]

#8589 has been merged! The DOM fixtures application is in place and functioning, now we need to focus on writing up all the fixture test cases and designing the UI for them

[jquense]

Some outstanding DOM issues, i'm personally familiar with are ('ill try and collect others as well)

1. Mouse enter and leave work consistently with disabled elements #5762 (I had an offline chat with @jimfb awhile ago about this and I believe it was left at "looks fine but we are all to scared to make such a big change to the mouse enter logic", so probably a good fit here.

2. Backport input fix #8575 Really is a collection of undocumented input behavior that needs documentation.

3. Change event does not fire in IE and Edge under certain circumstances #4672

[aweary]

A few issues that will make good fixtures: #7027, #4457, #8308

[jquense]

We still need to write the fixture components yeah? I could take a stab at that if its not in process somewhere

[aweary]

@jquense yupp, go for it! ping me when you've got a PR open and I'll review

[nhunzaker]

Started to catalog a check list of test cases in the issue description.

[nhunzaker]

I'm a bit stuck on this one:

Modify React flags via query strings, or some other mechanism to quickly switch between the Fiber and and standard DOM renderer.

Does React expose the ability to touch feature flags with the UMD build?

[nhunzaker]

@jquense @aweary I added your names next to the checklist of test cases to write based on the work progress. If I just created work for you, feel free to remove your names :).

[aweary]

@nhunzaker sounds good, I claimed the unclaimed ones 😃

[gaearon]

Can we close this now? 😃

[nhunzaker]

Yes. I think we certainly can.