Skipping based on PR labels to prevent hitting the rate limit

[achimnol]

What's the problem this feature will solve?

Since #9649, we can now skip the rest of a build job using special exit codes like 183.
However, it still occupies the rate limit as the build job needs to be started anyway as the skip decision is made inside the build script.

Describe the solution you'd like

I'd like to have a fast-path for skipping builds using metadata like GitHub PR's labels (e.g., continue or skip if a specific label exists / continue or skip if a specific label does not exist), before starting the build job, without occupying the rate limit.

[humitos]

Hi @achimnol! Thanks for your feature suggestion. We will discuss about it and see if we can find a place in our roadmap to prioritize this work.

Are you hitting the rate limit too frequently due to this issue? Would you please share what's the project URL where you are hitting this?

[achimnol]

It’s https://github.com/lablup/backend.ai . We have recently adopted https://graphite.dev and an automated backport PR generator, and now see increased rate of PR creation and updates. This makes it to hit the RTD build rate limit constantly, although most of builds are skipped.

[humitos]

Can you post the URL of the Read the Docs project?

[achimnol]

https://readthedocs.org/projects/sorna/
and
https://readthedocs.org/projects/sorna-ko

[achimnol]

We are observing that too many PR checks are pending due to the concurrency limit, while most of the jobs will be skipped.