build.yml: remove cached env if it appears invalid

[tripleee]

Closes: #7100

[makyen]

Unfortunately, I didn't see this while working on the same problem and running through test cycles in my fork. I like a combination of your solution, which only wipes the venv if the links are dead, and my solution, which retains the site_packages information for SD's dependencies while removing those which are dependent upon the Python minor version. However, your solution is safer, as it's possible for other packages to have a Python minor version dependency, or for new things to be added to the base Python virtual environment install. Overall, it's probably better to choose the safer version (i.e. yours).

[makyen]

After further testing, it looks like completely removing the venv from both GitHub Actions and CircleCI is a better way to go.

So, I'm going to close this.