Error exposing snapshot with velero CSI snapshot schedule

[nknoii]

What steps did you take and what happened:

1. Create a schedule with Velero CSI snapshot with data mover enabled.
2. Attempt to perform a backup using the created schedule.

What did you expect to happen:
The backup process should complete successfully without encountering any errors.

I encountered a similar issue as described here: Issue 7068. I tried to resolve it by increasing memory resources, but unfortunately, it didn't solve the problem for me.

If you are using earlier versions:
Please provide the output of the following commands (Pasting long output into a GitHub gist or other pastebin is fine.)

$ velero backup describe database-backup --details

Name:         database-backup
Namespace:    velero
Labels:       app.kubernetes.io/instance=velero
              velero.io/schedule-name=database-backup
              velero.io/storage-location=velero
Annotations:  argocd.argoproj.io/compare-options=IgnoreExtraneous
              argocd.argoproj.io/sync-options=Delete=false,Prune=false
              kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"velero.io/v1","kind":"Schedule","metadata":{"annotations":{"argocd.argoproj.io/compare-options":"IgnoreExtraneous","argocd.argoproj.io/sync-options":"Delete=false,Prune=false"},"creationTimestamp":null,"labels":{"app.kubernetes.io/instance":"velero"},"name":"database-backup","namespace":"velero"},"spec":{"schedule":"0 * * * *","template":{"csiSnapshotTimeout":"10m","hooks":{},"includedNamespaces":["database"],"metadata":{},"snapshotMoveData":true,"snapshotVolumes":true,"storageLocation":"velero","ttl":"72h","volumeSnapshotLocations":["velero-snapshot"]},"useOwnerReferencesInBackup":true},"status":{}}

  velero.io/resource-timeout=10m0s
  velero.io/source-cluster-k8s-gitversion=v1.26.13
  velero.io/source-cluster-k8s-major-version=1
  velero.io/source-cluster-k8s-minor-version=26

Phase:  PartiallyFailed (run `velero backup logs database-backup` for more information)


Errors:
  Velero:     <none>
  Cluster:    <none>
  Namespaces: <none>

Namespaces:
  Included:  database
  Excluded:  <none>

Resources:
  Included:        *
  Excluded:        <none>
  Cluster-scoped:  auto

Label selector:  <none>

Or label selector:  <none>

Storage Location:  velero

Velero-Native Snapshot PVs:  true
Snapshot Move Data:          true
Data Mover:                  velero

TTL:  336h0m0s

CSISnapshotTimeout:    10m0s
ItemOperationTimeout:  4h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2024-03-18 10:34:23 +0400 +0400
Completed:  2024-03-18 10:35:24 +0400 +0400

Expiration:  2024-04-01 10:34:22 +0400 +0400

Total items to be backed up:  85
Items backed up:              85

Backup Item Operations:
  Operation for persistentvolumeclaims database/postgres-db:
    Backup Item Action Plugin:  velero.io/csi-pvc-backupper
    Operation ID:               du-1182071e-8566-49e0-b626-392426a35bf8.aa317bac-9fc8-4f0eddf68
    Items to Update:
                           datauploads.velero.io velero/database-backup-7d8ll
    Phase:                 Failed
    Operation Error:       error to expose snapshot: error wait volume snapshot ready: error to get volumesnapshot database/velero-postgres-db-b8dlb: volumesnapshots.snapshot.storage.k8s.io "velero-postgres-db-b8dlb" not found
    Progress description:  Failed
    Created:               2024-03-18 10:34:48 +0400 +0400
    Started:               2024-03-18 10:34:48 +0400 +0400
    Updated:               2024-03-18 10:34:50 +0400 +0400

$ velero backup logs database-backup
https://gist.github.com/nknoii/c36367fadb9bdccd51c3e0bbca935904

Anything else you would like to add:

Environment:

• Velero version (use 6.0.0 (Helm chart)): App Version: 1.13.0, velero-plugin-for-aws:v1.9.0, velero-plugin-for-csi:v0.7.0
• Cloud provider or hardware configuration: DigitalOcean Cluster, s3 bucket
• OS (e.g. from /etc/os-release):

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• 👍 for "I would like to see this bug fixed as soon as possible"
• 👎 for "There are more important bugs to focus on right now"

[Lyndon-Li]

This looks more like #6852.
@nknoii Is this a consistent problem in your env? If so, please collect CSI external-snapshotter logs according to this comment after the error happens, and then collect Velero log bundle during the same period of time

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days. If a Velero team member has requested log or more information, please provide the output of the shared commands.

[github-actions]

This issue was closed because it has been stalled for 14 days with no activity.