Requirements with a specific worker model and a service

[b2p-fred]

I am trying to create a pipeline where my tests require to use some external services (e.g. database, Redis, ...).

I created a service requiring a worker model and 3 services:

version: v1.0
name: deploy-backend
description: Deploy the application backend
jobs:
- job: Setup the application package
  steps:
  - script:
    - curl -v -X GET http://my_nginx
  requirements:
  - model: eprotocole/epr-PHP
  - service:
      name: my_nginx
      value: nginx:1.11.1
  - service:
      name: my_pg
      value: postgres:9.5.3 POSTGRES_USER=myuser POSTGRES_PASSWORD=mypassword
  - service:
      name: my_redis
      value: redis:6.2.6

When I run a workflow including my deploy-backend pipeline, the result is:

Informations
✓ Job has been queued
2021-10-25 02:33:21 +02:00 Hatchery cds-hatchery-swarm-cdsinte starts spawn worker with model eprotocole/epr-PHP
2021-10-25 02:33:22 +02:00 Hatchery cds-hatchery-swarm-cdsinte starts docker pull redis:6.2.6...
2021-10-25 02:33:26 +02:00 ⚠ Hatchery cds-hatchery-swarm-cdsinte - docker pull redis:6.2.6 done with error: internal server error
2021-10-25 02:33:27 +02:00 Error while Hatchery cds-hatchery-swarm-cdsinte spawns worker with model eprotocole/epr-PHP after 4s, err:internal server error
2021-10-25 02:33:27 +02:00 The pipeline has been stopped by fmohier

It is not able to get the Docker image for the Redis service. One should note that there was no problem with the nginx and postgresql docker images ...

I tried with :

• different Docker images,
• different tags
• different tags for nginx and postgresql
  but I always have the same problem ... it looks like only the nginx and postgresql that are used as samples in the tutorials doc are accepted !

What about this ?

[Edit] After some more investigation I found out that the images used as a service are not pulled because of a refused authentication ...

2021-10-25 14:24:13 [ERROR] hatchery cds-hatchery-swarm-cdsinte cannot spawn worker eprotocole/epr-PHP for job 4563: Exec>startWorkerStarters>workerStarter>spawnWorkerForJob>SpawnWorker>createAndStartContainer>pullImage: internal server error (caused by: Unable to pull image redis:6.2.6 on sample-docker-engine: Error response from daemon: Head https://registry-1.docker.io/v2/library/redis/manifests/6.2.6: unauthorized: incorrect username or password) caller=github.com/ovh/cds/sdk/hatchery.spawnWorkerForJob goroutine=workerStarter-8 service=hatchery:swarm source_file=/tmp/790ad7b9a1a85167593986ed564384b7/run/sdk/hatchery/starter.go source_line=208 stack_trace=Error response from daemon: Head https://registry-1.docker.io/v2/library/redis/manifests/6.2.6: unauthorized: incorrect username or password

where should I define the credentials for this authentication ?

[b2p-fred]

As a workaround, I used a pipeline to login near my Docker registry and then pull the images I am interested in.

Nevertheless, it may be really interesting to use Docker images for services as it is implemented for worker models: allow to define a private registry.

[yesnault]

Hi @b2p-fred
Can you try to docker pull redis:6.2.6 on your docker host ? Is it ok?

It's possible to define a private registry on a worker model. Settings -> Worker model -> select type docker -> checked the Private registry.
But here, you don't need to use a private registry, you need to docker login on your docker host if you have an error with some docker limit pull.
And fyi, in the next release, it will be possible to set the docker credentials into the hatchery configuration (#5908)

[b2p-fred]

Hi @yesnault
For sure, the worker model can be an image pulled from a private repository; I do not have any problem with this.

But a service cannot be pulled from a private nor a public registry. As you suggested, I tried to use redis:6.2.6 as a service but I got an error message: Hatchery cds-hatchery-swarm-cdsinte - docker pull redis:6.2.6 done with error: internal server error.

BTW, it is not possible, IMHO, to declare a service requirement if the related Docker image is not yet pulled and available in the hatchery. Else, please, would you mind post an example ?

[yesnault]

This pipeline:

version: v1.0
name: issue5985
jobs:
- job: New Job
  steps:
  - script:
    - curl -v -X GET http://my_nginx
  requirements:
  - model: Debian10-Container
  - service:
      name: my_nginx
      value: nginx:1.11.1
  - service:
      name: my_pg
      value: postgres:9.5.3 POSTGRES_USER=myuser POSTGRES_PASSWORD=mypassword
  - service:
      name: my_redis
      value: redis:6.2.6

is valid.

Spawn infos are ok and job too:

Informations
✓ Job has been queued 2021-10-28 05:10:46 +02:00
Hatchery my-swarm starts spawn worker with model shared.infra/Debian10-Container 2021-10-28 05:10:48 +02:00
Hatchery my-swarm starts docker pull nginx:1.11.1... 2021-10-28 05:10:49 +02:00
Hatchery my-swarm docker pull nginx:1.11.1 done 2021-10-28 05:11:00 +02:00
Hatchery my-swarm starts docker pull postgres:9.5.3... 2021-10-28 05:11:01 +02:00
Hatchery my-swarm docker pull postgres:9.5.3 done 2021-10-28 05:11:04 +02:00
Hatchery my-swarm starts docker pull redis:6.2.6... 2021-10-28 05:11:05 +02:00
Hatchery my-swarm docker pull redis:6.2.6 done 2021-10-28 05:11:10 +02:00
Job 15149755 has been taken by worker shared-infra-debian10-container-crazy-and-adoring-booth 2021-10-28 05:11:18 +02:00
Worker shared-infra-debian10-container-crazy-and-adoring-booth version:0.49.0-118-g4e320d278.16695 os:linux arch:amd64 2021-10-28 05:11:18 +02:00
✓ Worker shared-infra-debian10-container-crazy-and-adoring-booth finished working on this job

Until the next release, if you need to run a service pre-requisite using a private registry, you have to docker login to the private registry on your docker host, this should work.

But, here, you don't use private registry and you have an error, some checks to do:

• do you see something in the hatchery logs ?
• can you check the logs of the redis containers spawned ?
• same question for the worker model's container.

[b2p-fred]

Thank you for this information. I made some more tests and I confirm that I cannot Docker pull public images. I will investigate with the hatchery log.

[yesnault]

Feel free to reopen if needed.