Firefox crashes in Docker VM

[ghost]

Describe the bug

When starting firefox or lxterminal they crash with an MIT-SHM error.

GNS3 version and operating system:

• OS: Debian Linux
• GNS3 version 2.2.32
• GNS3 VM running on ARM64 Alpine Linux v3.16.0 emulated by Qemu 5.2
• All servers installed by "pip3 install gns3server"

To Reproduce

Steps to reproduce the behavior:

1. Create docker VM with image ghcr.io/b-ehlers/webterm (VNC console) in ARM64 Alpine GNS3VM
2. Create project and add this webterm
3. Configure start command of webterm to sh
4. Start webterm and open VNC console and auxiliary console
5. Start lxterminal or firefox in auxiliary console
6. See error

Error message

/ # lxterminal

(lxterminal:3760): Gdk-ERROR **: 06:47:02.721: The program 'lxterminal' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadValue (integer parameter out of range for operation)'.
  (Details: serial 188 error_code 2 request_code 130 (MIT-SHM) minor_code 5)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the GDK_SYNCHRONIZE environment
   variable to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)
Trace/breakpoint trap (core dumped)

Additional context

I don't have this issue, when running firefox directly on my x86 Debian Linux. There are a couple of differences between these two environments:

• qemu-system-aarch64 v5.2 vs. bare metal Intel processor
• Alpine Linux v3.16.0 vs. Debian 11 (Bullseye)
• x11vnc+Xvfb vs. Xtigervnc

But nevertheless this MIT-SHM error seems to be a common issue, when running GUI applications in Docker. Good information about that can be found in MIT-SHM error solutions.

I tested both suggestions and both are fixing my issue:

Disable IPC namespacing

Using the docker option --ipc=host fixes this issue, but it decreases the docker isolation. So this is not optimal.

The patch:

diff --git a/gns3server/compute/docker/docker_vm.py b/gns3server/compute/docker/docker_vm.py
index a7d5c322..9080205f 100644
--- a/gns3server/compute/docker/docker_vm.py
+++ b/gns3server/compute/docker/docker_vm.py
@@ -343,6 +343,7 @@ class DockerVM(BaseNode):
             "HostConfig": {
                 "CapAdd": ["ALL"],
                 "Privileged": True,
+                "IpcMode": "host",
                 "Binds": self._mount_binds(image_infos),
             },
             "Volumes": {},

Running an additional X server with extension MIT-SHM disabled

GNS3 already starts a new X server for every docker VM. What needs to be done, is to disable MIT-SHM for these new X servers by adding the option -extension MIT-SHM. Here the patch:

diff --git a/gns3server/compute/docker/docker_vm.py b/gns3server/compute/docker/docker_vm.py
index a7d5c322..2d169a43 100644
--- a/gns3server/compute/docker/docker_vm.py
+++ b/gns3server/compute/docker/docker_vm.py
@@ -584,6 +584,7 @@ class DockerVM(BaseNode):
         if tigervnc_path:
             with open(os.path.join(self.working_dir, "vnc.log"), "w") as fd:
                 self._vnc_process = await asyncio.create_subprocess_exec(tigervnc_path,
+                                                                         "-extension", "MIT-SHM",
                                                                          "-geometry", self._console_resolution,
                                                                          "-depth", "16",
                                                                          "-interface", self._manager.port_manager.console_host,
@@ -595,8 +596,9 @@ class DockerVM(BaseNode):
         else:
             if restart is False:
                 self._xvfb_process = await asyncio.create_subprocess_exec("Xvfb",
-                                                                          "-nolisten",
-                                                                          "tcp", ":{}".format(self._display),
+                                                                          "-nolisten", "tcp",
+                                                                          "-extension", "MIT-SHM",
+                                                                          ":{}".format(self._display),
                                                                           "-screen", "0",
                                                                           self._console_resolution + "x16")
 
@@ -606,6 +608,7 @@ class DockerVM(BaseNode):
                                                                          "-forever",
                                                                          "-nopw",
                                                                          "-shared",
+                                                                         "-noshm",
                                                                          "-geometry", self._console_resolution,
                                                                          "-display", "WAIT:{}".format(self._display),
                                                                          "-rfbport", str(self.console),

The Xvfb option -nolisten tcp belong together, so I moved them on the same line.

[ghost]

Just to make it clear, it's sufficient to implement one of my proposals. I suggest to use the second one.

[grossmj]

Just to make it clear, it's sufficient to implement one of my proposals. I suggest to use the second one.

Thanks, that is my intention :)

[ghost]

Just a comment: The changes disable MIT-SHM not only for XtigerVNC (as noted in the commit message), but also when using x11vnc+Xvfb. In fact I successfully tested it on the ARM64 alpine GNS3VM with x11vnc+Xvfb. So both variants should be fine now.