Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-11801. Logs missing if kubernetes check fails before tests #7496

Merged
merged 2 commits into from
Jan 3, 2025

Conversation

adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

Logs are not collected if Kubernetes check fails before starting tests (due to set -e exiting early).

On master prior to HDDS-11718 kubernetes check was failing (silently):
https://github.com/apache/ozone/actions/runs/12033431383/job/33548517871

Excerpt from Kubernetes check output:

100 'grep_log scm-0 SCM exiting safe mode.' is failed...
...
Process completed with exit code 1.

Artifact has no logs from the cluster.

https://issues.apache.org/jira/browse/HDDS-11801

How was this patch tested?

Tested this change on top of cc1a374:
https://github.com/adoroszlai/ozone/actions/runs/12048774547

Excerpt from Kubernetes check output:

100 'grep_log scm-0 SCM exiting safe mode.' is failed...
...
ERROR: Test execution of getting-started is FAILED!!!!

Artifact has logs:

$ unzip -t kubernetes.zip 
Archive:  kubernetes.zip
    testing: getting-started/pod-datanode-0.log   OK
    testing: getting-started/pod-datanode-1.log   OK
    testing: getting-started/pod-datanode-2.log   OK
    testing: getting-started/pod-om-0.log   OK
    testing: getting-started/pod-s3g-0.log   OK
    testing: getting-started/pod-scm-0-init.log   OK
    testing: getting-started/pod-scm-0.log   OK
    testing: minikube/pod-datanode-0.log   OK
    testing: minikube/pod-datanode-1.log   OK
    testing: minikube/pod-datanode-2.log   OK
    testing: minikube/pod-om-0.log    OK
    testing: minikube/pod-s3g-0.log   OK
    testing: minikube/pod-scm-0-init.log   OK
    testing: minikube/pod-scm-0.log   OK
    testing: output.log               OK
    testing: ozone/pod-datanode-0.log   OK
    testing: ozone/pod-datanode-1.log   OK
    testing: ozone/pod-om-0.log       OK
    testing: ozone/pod-s3g-0.log      OK
    testing: ozone/pod-scm-0-init.log   OK
    testing: ozone/pod-scm-0.log      OK
    testing: ozone-dev/pod-datanode-0.log   OK
    testing: ozone-dev/pod-datanode-1.log   OK
    testing: ozone-dev/pod-datanode-2.log   OK
    testing: ozone-dev/pod-jaeger-0.log   OK
    testing: ozone-dev/pod-om-0.log   OK
    testing: ozone-dev/pod-prometheus-548f95bb6b-tr9b9.log   OK
    testing: ozone-dev/pod-s3g-0.log   OK
    testing: ozone-dev/pod-scm-0-init.log   OK
    testing: ozone-dev/pod-scm-0.log   OK
    testing: ozone-ha/pod-datanode-0.log   OK
    testing: ozone-ha/pod-datanode-1.log   OK
    testing: ozone-ha/pod-datanode-2.log   OK
    testing: ozone-ha/pod-om-0.log    OK
    testing: ozone-ha/pod-s3g-0.log   OK
    testing: ozone-ha/pod-scm-0-bootstrap.log   OK
    testing: ozone-ha/pod-scm-0-init.log   OK
    testing: ozone-ha/pod-scm-0.log   OK
    testing: ozone-ha/pod-scm-1-bootstrap.log   OK
    testing: ozone-ha/pod-scm-1-init.log   OK
    testing: ozone-ha/pod-scm-1.log   OK
    testing: ozone-ha/pod-scm-2-bootstrap.log   OK
    testing: ozone-ha/pod-scm-2-init.log   OK
    testing: ozone-ha/pod-scm-2.log   OK
    testing: summary.txt              OK
No errors detected in compressed data of kubernetes.zip.

CI (after HDDS-11718):
https://github.com/adoroszlai/ozone/actions/runs/12049809854

@adoroszlai adoroszlai self-assigned this Nov 27, 2024
@adoroszlai adoroszlai added the CI label Nov 27, 2024
Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a Kubernetes cluster, but the script change looks good to me.

@adoroszlai adoroszlai merged commit 99fd5fc into apache:master Jan 3, 2025
15 checks passed
@adoroszlai adoroszlai deleted the HDDS-11801 branch January 3, 2025 12:51
@adoroszlai
Copy link
Contributor Author

Thanks @jojochuang for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants