Feature/add cleanup non recoverable pod statuses

[pvditt]

Tracking issue

Closes: #4198

Why are the changes needed?

Pods hang indefinitely

What changes were proposed in this pull request?

Cleanup pods on failure

How was this patch tested?

• Updated unit testing

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

Related PRs

Docs link

[codecov]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (b562536) 59.02% compared to head (2d76091) 59.04%.

Files	Patch %	Lines
...lyteplugins/go/tasks/pluginmachinery/core/phase.go	0.00%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4607      +/-   ##
==========================================
+ Coverage   59.02%   59.04%   +0.01%     
==========================================
  Files         622      622              
  Lines       52685    52687       +2     
==========================================
+ Hits        31098    31109      +11     
+ Misses      19105    19098       -7     
+ Partials     2482     2480       -2     

Flag	Coverage Δ
unittests	59.04% <36.36%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.