Interop testing ideas and current limitations

[dryajov]

This is an issue to start a discussion around js-ipfs interop testing across different environments.

Our current testing infrastructure is a bit cumbersome to use for browser interop testing. It's not possible to signal starting and stopping nodes from within the karma ran browser tests, hence we relay on externally started nodes on well known ports, which work in some cases, but are not of great use when a clean env is required before each test run - i.e. with circuit-relay, once a connection is made over a circuit, forcing tests to use another circuit is not possible in certain cases.

Ideally, I would like to be able for interop tests to be reusable across different runtimes - node, browser, electron, etc... However, that doesn't seem to be possible right now, at least not for the browser. @dignifiedquire I might be wrong on this one, but what I'm talking about would require karma having the ability of spawning external procs from within the tests, and thats not currently possible as far as I can tell. Moreover, I don't want to come up with an ad-hoc solution that will work for my case but might not be the right solution for testing js-ipfs across the board.

I also wanted to collect ideas as to how to structure our testing better with the new version of aegir that brings jest support.

[daviddias]

Our current testing infrastructure is a bit cumbersome to use for browser interop testing.

Now it has improved since we use AEgir as well

It's not possible to signal starting and stopping nodes from within the karma ran browser tests,

Coming up with ipfs/js-ipfsd-ctl#176 🚢

I would like to be able for interop tests to be reusable across different runtimes - node, browser, electron, etc.

👍🏽 Coming up with ipfs/aegir#169, @mkg20001 what's the status on that PR?

[mkg20001]

@diasdavid Don't even know if the changes so far work. A lot of work needs to be done, especially to remove all the duplicated code.

[daviddias]

@dryajov can we move this discussion to https://github.com/ipfs/interop? I'm sure @victorbjelkholm will also have some thoughts on it :)