RTNETLINK answers: Operation not permitted / Cannot ioctl TUNSETIFF tun

[Sulter]

Describe the problem
The stack doesn't start due to the error in the title.
Please note that I already have the pi-hole in docker in swarm mode as well link, and that works just fine with the cap_add: net_admin. That's why I'm suspecting a possible bug with docker-transmission-openvpn, but this might also be a configuration error.

Add your docker run command

version: '3.3'
services:
    transmission-openvpn:
        volumes:
            - '/srv/dev-disk-by-label-mediaDisk/Film:/data'
        environment:
            - OPENVPN_PROVIDER=GHOSTPATH
            - OPENVPN_CONFIG=Germany-Frankfurt
            - OPENVPN_USERNAME=XXXXXXXX
            - OPENVPN_PASSWORD=YYYYYYYY
            - WEBPROXY_ENABLED=false
            - LOCAL_NETWORK=192.168.1.0/24
        cap_add:
            - NET_ADMIN
        logging:
            driver: json-file
            options:
                max-size: 10m
        ports:
            - '9091:9091'
        image: haugene/transmission-openvpn

Logs

Starting container with revision: baed33520630e88ba34842cab05f2cd3fd6cf28f,
Creating TUN device /dev/net/tun,
Using OpenVPN provider: GHOSTPATH,
Starting OpenVPN using config Germany-Frankfurt.ovpn,
Setting OpenVPN credentials...,
adding route to local network 192.168.1.0/24 via 172.18.0.1 dev eth2,
RTNETLINK answers: Operation not permitted,
Sun Nov 22 07:09:52 2020 OpenVPN 2.4.9 armv7-alpine-linux-musleabihf [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020,
Sun Nov 22 07:09:52 2020 library versions: OpenSSL 1.1.1g  21 Apr 2020, LZO 2.10,
Sun Nov 22 07:09:52 2020 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts,
Sun Nov 22 07:09:52 2020 TCP/UDP: Preserving recently used remote address: [AF_INET]139.162.159.184:443,
Sun Nov 22 07:09:52 2020 UDP link local: (not bound),
Sun Nov 22 07:09:52 2020 UDP link remote: [AF_INET]139.162.159.184:443,
Sun Nov 22 07:09:52 2020 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this,
Sun Nov 22 07:09:52 2020 [VPN] Peer Connection Initiated with [AF_INET]139.162.159.184:443,
Sun Nov 22 07:09:53 2020 ERROR: Cannot ioctl TUNSETIFF tun: Operation not permitted (errno=1),
Sun Nov 22 07:09:53 2020 Exiting due to fatal error,

Host system:
odroidxu4, Armbian 20.08.17 Buster with Linux 5.4.72-odroidxu4
Docker version 19.03.13, build 4484c46

[clement-z]

Note that you also have an error before the one you highlight.

[...]
adding route to local network 192.168.1.0/24 via 172.18.0.1 dev eth2,
RTNETLINK answers: Operation not permitted,
[...]
Sun Nov 22 07:09:53 2020 ERROR: Cannot ioctl TUNSETIFF tun: Operation not permitted (errno=1),
Sun Nov 22 07:09:53 2020 Exiting due to fatal error,

Running with NET_ADMIN should be enough to modify routes and configure interfaces.

Some ideas off the top of my head:

1. Can you make sure you recreated the container (docker-compose up -d -force-recreate).

2. If unsuccessful, can you try whether it works if you use privileged: true.

3. If privileged works but NET_ADMIN doesn't, can you add the following to your transmission-openvpn service

   entrypoint: dumb-init /bin/bash -i
   stdin_open: true
   tty: true

   then run and show the ouptut of:

   • docker exec -it container_name bash -c 'apk add libcap; getpcaps 1'
   • docker exec -it container_name bash -c 'ip a; ip r'

[Sulter]

@clement-z thanks for taking a look at it.
It seems like running the container directly with docker run with the exact same options, works perfectly fine.
I only get the above error when deploying it as a stack. I also first now noticed that when I try to run it as a stack, I get the:
Ignoring unsupported options: cap_add

Searching that, it looks like docker, in swarm mode, doesn't support cap_add at all?
Is it then possible to run in non-swarm mode and use the v. 3.3 compose from the readme? - I get an error when I try to.

So I my question is whatever my docker in swarm mode is misconfigured to be used as stack as showed in the readme, or if we should update the readme with possible additional steps or completely remove the stack?
There might be other options, but my understanding of docker, the different compose versions and swarm mode is far too limited to see other solutions.

[haugene]

You should be able to run docker-compose without swarm. That is what I do and I think a lot of people use that option. It's not deployed as a stack though. I think stack is a swarm-thing(?). I think I've seen in some other issue here that someone said cap_add was in an experimental version of Docker or something so it might be coming soon.

Disclaimer. I've worked a lot with Docker, but not with stacks so I might be missing something here.

[clement-z]

@Sulter I am not familiar with swarm mode, but as @haugene mentioned, it seems it will be available in the next release (docker/cli#2687).

However, if you run on a single host, I think docker-compose is more appropriate? And I can confirm it works too! 😄

[davehope]

Just to chime in on this issue, running docker on Debian x64 stable.

# docker -v
Docker version 20.10.1, build 831ebea

With cap_add in my compose-file, the capabilities don't seem to come across yet (Docker swarm):

bash-5.0# apk add libcap; getpcaps 1
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
(1/1) Installing libcap (2.27-r0)
Executing busybox-1.31.1-r19.trigger
OK: 99 MiB in 69 packages
Capabilities for `1': = cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap+eip

However, I can manually set them on the service and validate with getpcaps. The VPN then comes up as expected:

docker service update xxx --cap-add NET_ADMIN

This can then be confirmed (from the docker perspective):

docker inspect xxx | jq '.[0].Spec.TaskTemplate.ContainerSpec.CapabilityAdd'

At the time of writing, I'm using docker-ce/buster,now 5:20.10.1~3-0~debian-buster amd64 [installed] and deploying my stacks using portainer.

I may be doing something wrong - interested to hear if anyone has something working with swarm from docker 20.10.1 onwards.

[davehope]

As a follow-up to my past comment, the issue doesn't occur with docker swarm in 20.10.1 providing the compose-file is deployed using docker stack deploy --compose-file transmission-openvpn.yml transmission-openvpn rather than relying on something like Portainer to deploy the same compose-file.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[stale]

Feel free to re-open this issue if you think it deserves another look.