Visors are intermittently disconnecting to the hypervisor

[Senyoret1]

Describe the bug
When using the hypervisor to get info about the visors, it is very common to find errors due to the visors being intermittently disconnected from the hypervisor.

Environment information:

• OS: Linux (Ubuntu 18.04.1)
• Platform: Linux 4.15.0-65-generic x86_64

Steps to Reproduce
If you call GET http://{localIp}:8080/api/visors frequently, some times the response will have one or more visors with most fields empty.

Also, calling any of the API endpoints for getting info about a specific visor sometimes results in geting unexpected EOF or connection is shut down.

Actual behavior
The API is returning invalid responses when the connection to the visors is lost for a brief period of time.

The problem is frequent enough to be quite annoying in the manager UI.

Expected behavior
The API should return the expected responses.

Additional context

Possible implementation
If there is any serious complication for implementing a solution in the hypervisor, the client could implement something like a “noice cancellation” procedure to detect in which cases the hypervisor is just having a temporary disconnection. This would sometimes make the UI slower but should work.

If this is going to be done, it would be good to document the need to do so in some location related to the API, including ways to detect the disconnection and the amount of time in which a reconnection could be expected, so anyone using the API is aware of the need to do something similar.

[Senyoret1]

I recently noticed that when the visor is reconnected, it shows a different port number. In fact, the port number keeps increasing over time, after each disconnection and reconnection.

[Senyoret1]

This was initially tested using the mainnet-milestone2 branch. The master branch does not have the error, but that may be due to the usage of dmsg in the hypervisor in the mainnet-milestone2 branch.

[jdknives]

Seems to be fixed.