How to use Dragonfly with CRI-O?

[orange64]

Question

I want to use Dragonfly in my k8s cluster,which used CRI-O container engine:

NAME      STATUS   ROLES    AGE   VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION              CONTAINER-RUNTIME
node1      Ready    <none>   21h   v1.16.0      xxx        <none>        CentOS Linux 7 (Core)   5.7.7-1.el7.elrepo.x86_64   cri-o://1.16.6

I tried to modify some configurations, but when I pulled the image, it did not use dragonfly.
Where is the problem？

I want to pull images from harbor and docker.io,quay.mirrors.ustc.edu.cn,hub-mirror.c.163.com.etc

dfdaemon.yml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels: 
    k8s-app: dfdaemon
  name: dfdaemon
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: dfdaemon
  template:
    metadata:
      labels:
        app: dfdaemon
    spec:
      containers:
      - name: dfdaemon
        image: dragonflyoss/dfclient:1.0.3
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: 250m
        volumeMounts:
        - mountPath: /etc/dragonfly/dfdaemon.yml
          subPath: dfdaemon.yml
          name: dfconf
        - mountPath: /root/.small-dragonfly
          name: data 
        - mountPath: /certs
          name: certs
        - mountPath: /etc/localtime
          name: localtime
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      restartPolicy: Always
      tolerations: 
      - effect: NoExecute
        operator: Exists
      - effect: NoSchedule
        operator: Exists
      volumes:
      - name: dfconf
        configMap:
          name: dfdaemon-conf
          defaultMode: 384
      - name: data
        hostPath: 
          path: /home/eecdn/.small-dragonfly
          type: DirectoryOrCreate
      - name: certs
        hostPath:
           path: /data/dfcert
           type: DirectoryOrCreate 
      - name: localtime
        hostPath: 
          path: /etc/localtime

---
apiVersion: v1
kind: ConfigMap
metadata:
  labels: 
    k8s-app: dfdaemon
  name: dfdaemon-conf
  namespace: kube-system
data:
  dfdaemon.yml: |-
    # proxy all http image layer download requests with dfget
    proxies:
      - regx: blobs/sha256.*
    hijack_https:
      cert: /certs/df.crt
      key: /certs/df.key
      hosts:
        - regx: myharbor.test.com:1443
          insecure: true
    supernodes:                       
      - supernode-ip

/etc/containers/registries.conf : I added [registries.insecure] and [registries.mirror] as following
(referring this issue: cri-o/cri-o#372)

unqualified-search-registries = ['myharbor.test.com:1443','hub-mirror.c.163.com','docker.io','quay.mirrors.ustc.edu.cn', 'registry.access.redhat.com', 'registry.fedoraproject.org', 'quay.io', 'registry.centos.org']

[[registry]]
prefix = "myharbor.test.com:1443"
insecure = true
blocked = false
location = "localhost:65001"

[[registry]]
prefix = "docker.io/library"
insecure = true
blocked = false
location = "localhost:65001/library"

[[registry.mirror]]
location = "localhost:65001"
insecure = true

/usr/lib/systemd/system/crio.service : I added

Environment="HTTP_PROXY=http://127.0.0.1:65001"
Environment="HTTPS_PROXY=http://127.0.0.1:65001"

[orange64]

When I execute "crictl pull busybox",I can see that there are packets passing 127.0.0.1:65001,which means http_proxy is working:

[ecf@node1 ~]$ sudo tcpdump -i lo -nn -A port 65001
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
17:06:22.484033 IP 127.0.0.1.50746 > 127.0.0.1.65001: Flags [S], seq 239256780, win 65495, options [mss 65495,sackOK,TS val 2426755491 ecr 0,nop,wscale 7], length 0
E..<.>@.@.#|.........:...B...........0.........
..Y.........
17:06:22.484058 IP 127.0.0.1.65001 > 127.0.0.1.50746: Flags [S.], seq 2124295845, ack 239256781, win 65483, options [mss 65495,sackOK,TS val 2426755491 ecr 2426755491,nop,wscale 7], length 0
E..<..@.@.<............:~....B.......0.........
..Y...Y.....
17:06:22.484079 IP 127.0.0.1.50746 > 127.0.0.1.65001: Flags [.], ack 1, win 512, options [nop,nop,TS val 2426755491 ecr 2426755491], length 0
E..4.?@.@.#..........:...B..~........(.....
..Y...Y.
17:06:22.484220 IP 127.0.0.1.50746 > 127.0.0.1.65001: Flags [P.], seq 1:110, ack 1, win 512, options [nop,nop,TS val 2426755491 ecr 2426755491], length 109
E....@@.@.#..........:...B..~..............
..Y...Y.CONNECT hub-mirror.c.163.com:443 HTTP/1.1
Host: hub-mirror.c.163.com:443
User-Agent: Go-http-client/1.1

.....

[jim3ma]

Please, exec into dfclient container, cat /etc/dragonfly/dfdaemon.yml

[orange64]

Please, exec into dfclient container, cat /etc/dragonfly/dfdaemon.yml

dfdaemon.yml: |-
# proxy all http image layer download requests with dfget
proxies:
- regx: blobs/sha256.*
hijack_https:
cert: /certs/df.crt
key: /certs/df.key
hosts:
- regx: myharbor.test.com:1443
insecure: true
supernodes:
- supernode-ip

[jim3ma]

Enable verbose like this, and pull image again

proxies:
- regx: blobs/sha256.*
dfget_flags:
- "--node"
- "127.0.0.1"
# add this line
verbose: true

then take a look on dfdaemon.log