No such container: matrix-user-verification-service

[ZelChief]

Describe the bug

How to debug error No such container: matrix-user-verification-service?

TASK [galaxy/com.devture.ansible.role.systemd_service_manager : Ensure systemd services are started]******************
changed: [matrix.mydomain.com] => (item={'name': 'matrix-postgres.service', 'priority': 500, 'groups': ['matrix', 'postgres']})
failed: [matrix.mydomain.com] (item={'name': 'matrix-user-verification-service.service', 'priority': 800, 'groups': ['matrix', 'matrix-user-verification-service']}) => changed=false 
  ansible_loop_var: item
  item:
    groups:
    - matrix
    - matrix-user-verification-service
    name: matrix-user-verification-service.service
    priority: 800
  msg: |-
    Unable to start service matrix-user-verification-service.service: Job for matrix-user-verification-service.service failed because the control process exited with error code.
    See "systemctl status matrix-user-verification-service.service" and "journalctl -xe" for details.
changed: [matrix.mydomain.com] => (item={'name': 'matrix-synapse.service', 'priority': 1000, 'groups': ['matrix', 'homeservers', 'synapse']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-synapse-reverse-proxy-companion.service', 'priority': 1500, 'groups': ['matrix', 'homeservers', 'synapse', 'reverse-proxies']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-client-element.service', 'priority': 2000, 'groups': ['matrix', 'clients', 'element', 'client-element']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-mailer.service', 'priority': 2000, 'groups': ['matrix', 'mailer']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-container-socket-proxy.service', 'priority': 2900, 'groups': ['matrix', 'reverse-proxies', 'container-socket-proxy']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-nginx-proxy.service', 'priority': 3000, 'groups': ['matrix', 'nginx', 'reverse-proxies']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-traefik.service', 'priority': 3000, 'groups': ['matrix', 'traefik', 'reverse-proxies']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-traefik-certs-dumper.service', 'priority': 3500, 'groups': ['matrix', 'traefik-certs-dumper']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-coturn.service', 'priority': 4000, 'groups': ['matrix', 'coturn']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-jitsi-prosody.service', 'priority': 4000, 'groups': ['matrix', 'jitsi', 'jitsi-prosody']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-synapse-admin.service', 'priority': 4000, 'groups': ['matrix', 'synapse-admin']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-jitsi-jicofo.service', 'priority': 4100, 'groups': ['matrix', 'jitsi', 'jitsi-jicofo']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-jitsi-jvb.service', 'priority': 4100, 'groups': ['matrix', 'jitsi', 'jitsi-jvb']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-jitsi-web.service', 'priority': 4200, 'groups': ['matrix', 'jitsi', 'jitsi-web']})
changed: [matrix.mydomain.com] => (item={'name': 'matrix-coturn-reload.timer', 'priority': 5000, 'groups': ['matrix', 'coturn']})

On VM: Error response from daemon: No such container: matrix-user-verification-service

$ sudo journalctl -xe
░░ A start job for unit matrix-user-verification-service.service has begun execution.
░░ 
░░ The job identifier is 1324.
Oct 29 23:41:15 matrix matrix-user-verification-service[5342]: Error response from daemon: No such container: matrix-user-verification-service
Oct 29 23:41:15 matrix systemd[1]: matrix-user-verification-service.service: Control process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStartPre= process belonging to unit matrix-user-verification-service.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Oct 29 23:41:15 matrix systemd[1]: matrix-user-verification-service.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit matrix-user-verification-service.service has entered the 'failed' state with result 'exit-code'.
Oct 29 23:41:15 matrix systemd[1]: Failed to start Matrix User Verification Service.
░░ Subject: A start job for unit matrix-user-verification-service.service has failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit matrix-user-verification-service.service has finished with a failure.
░░ 
░░ The job identifier is 1324 and the job result is failed.
Oct 29 23:41:15 matrix sudo[5309]: pam_unix(sudo:session): session closed for user root
Oct 29 23:41:25 matrix systemd[1]: run-docker-runtime\x2drunc-moby-023fb4547f9c5b4c78a9c1e1d4420a6c7d9027ee95674870312bc282ad582817-runc.IVLief.mount: Succeeded.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit run-docker-runtime\x2drunc-moby-023fb4547f9c5b4c78a9c1e1d4420a6c7d9027ee95674870312bc282ad582817-runc.IVLief.mount has successfully entered the 'dead' state.
Oct 29 23:41:25 matrix systemd[4525]: run-docker-runtime\x2drunc-moby-023fb4547f9c5b4c78a9c1e1d4420a6c7d9027ee95674870312bc282ad582817-runc.IVLief.mount: Succeeded.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit UNIT has successfully entered the 'dead' state.

How to debug problem?

To Reproduce
My vars.yml file looks like this:

---

matrix_domain: mydomain.com
matrix_homeserver_implementation: synapse
matrix_homeserver_generic_secret_key: 'w9JTzANU343g434gGE5XsBR3yfrOU0ye'
matrix_playbook_reverse_proxy_type: playbook-managed-traefik

devture_traefik_config_certificatesResolvers_acme_email: '[email protected]'
devture_postgres_connection_password: 'TthWt245tc42t63aFkwlUH'

jitsi_enabled: true
jitsi_enable_auth: true
jitsi_enable_guests: false
jitsi_auth_type: matrix

matrix_user_verification_service_enabled: true

jitsi_web_custom_config_extension: |
  config.enableLayerSuspension = true;
  config.disableAudioLevels = true;
  config.channelLastN = 4;

jitsi_web_config_resolution_width_ideal_and_max: 480
jitsi_web_config_resolution_height_ideal_and_max: 240

devture_traefik_config_certificatesResolvers_acme_use_staging: true

matrix_synapse_federation_domain_whitelist:
- mydomain1.com
- mydomain2.com

matrix_synapse_admin_enabled: true
matrix_nginx_proxy_proxy_matrix_client_api_forwarded_location_synapse_admin_api_enabled: true

Matrix Server:

• OS: Debian GNU/Linux 11 (bullseye)
• Architecture: amd64
• Python 3.9.2

Ansible:
I run Ansible on macOS 14.0 (23A344).

% ansible --version
ansible [core 2.14.1]
  config file = /Users/User1/matrix/ansible.cfg
  configured module search path = ['/Users/User1/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/User1/Library/Python/3.9/lib/python/site-packages/ansible
  ansible collection location = /Users/User1/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/User1/Library/Python/3.9/bin/ansible
  python version = 3.9.6 (default, Aug 11 2023, 19:44:49) [Clang 15.0.0 (clang-1500.0.40.1)] (/Library/Developer/CommandLineTools/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

[spantaleev]

Perhaps you have enabled the service (by adding matrix_user_verification_service_enabled: true to your vars.yml file), and are trying to do --tags=start, but.. did not do --tags=install-all (or --tags=setup-all) to actually install the service first?

[spantaleev]

Oh, wait.. Reading your issue description again, the service seems to exist and tries to start, but fails.

Try journalctl -fu matrix-user-verification-service.service - hopefully this will give you more details about why it fails. compared to journalctl -xe

[thiras]

I'm having exactly same problem. Here is the journalctl -fu matrix-user-verification-service.service output;

Oct 30 11:15:32 matrix.example.com systemd[1]: matrix-user-verification-service.service: Scheduled restart job, restart counter is at 63.
Oct 30 11:15:32 matrix.example.com systemd[1]: Stopped matrix-user-verification-service.service - Matrix User Verification Service.
Oct 30 11:15:32 matrix.example.com systemd[1]: Starting matrix-user-verification-service.service - Matrix User Verification Service...
Oct 30 11:15:32 matrix.example.com matrix-user-verification-service[22462]: Error response from daemon: No such container: matrix-user-verification-service
Oct 30 11:15:32 matrix.example.com systemd[1]: matrix-user-verification-service.service: Control process exited, code=exited, status=1/FAILURE
Oct 30 11:15:32 matrix.example.com systemd[1]: matrix-user-verification-service.service: Failed with result 'exit-code'.
Oct 30 11:15:32 matrix.example.com systemd[1]: Failed to start matrix-user-verification-service.service - Matrix User Verification Service.

Probably related to #2847

[ZelChief]

spantaleev, thanks for the quick response! Done:

$ sudo journalctl -fu matrix-user-verification-service.service
-- Journal begins at Sat 2023-10-28 19:12:45 UTC. --
Oct 30 18:07:36 matrix systemd[1]: matrix-user-verification-service.service: Control process exited, code=exited, status=1/FAILURE
Oct 30 18:07:36 matrix systemd[1]: matrix-user-verification-service.service: Failed with result 'exit-code'.
Oct 30 18:07:36 matrix systemd[1]: Failed to start Matrix User Verification Service.
Oct 30 18:08:07 matrix systemd[1]: matrix-user-verification-service.service: Scheduled restart job, restart counter is at 1.
Oct 30 18:08:07 matrix systemd[1]: Stopped Matrix User Verification Service.
Oct 30 18:08:07 matrix systemd[1]: Starting Matrix User Verification Service...
Oct 30 18:08:07 matrix matrix-user-verification-service[5104]: Error response from daemon: No such container: matrix-user-verification-service
Oct 30 18:08:07 matrix systemd[1]: matrix-user-verification-service.service: Control process exited, code=exited, status=1/FAILURE
Oct 30 18:08:07 matrix systemd[1]: matrix-user-verification-service.service: Failed with result 'exit-code'.
Oct 30 18:08:07 matrix systemd[1]: Failed to start Matrix User Verification Service.

[ZelChief]

@thiras, your fix has helped! Thx 👍