Action for asking ChatGPT to review PRs (OSOE-499)

[Piedone]

Let's create a proof of concept that integrates with ChatGPT and asks it to review the PR. This could run specific prompts for each code file, perhaps for the whole file or just the diff like this:

Or:

Now, this would be experimental, and ChatGPT can provide useless or incorrect suggestions too (there are examples of both above). But, it's interesting to see if it makes sense to use for this.

• Figure out which prompts actually make sense. These might depend on the language of the code file and other parameters. Some more ones for inspiration: "Improve the performance of this code", "Simplify this code", "Write unit test cases for this code", "Write documentation for this code". Also see this thread for some tips.
• See if you can add context for ChatGPT for it to provide better feedback.
• Check whether feeding it the whole code file or just the changed section would make sense. Note just the changed section might be too small for it to make sense alone.
• Implement this in a new action, that's called by a new reusable workflow.
• Run it for OSOCE PRs, but for now, only in an opt-in way, when the "ask-chatgpt-for-feedback" label is present. It should be triggered on pushes to PRs as well as during labeling (so when you add the label after a push, it'll run too).

Jira issue

[I3undy]

As an initial thought, I signed up the for the API and the premium waitlists because there are more outages than normal operations this week.

[Piedone]

Perhaps we can use ChatGPT as an Azure service instead? https://azure.microsoft.com/en-us/blog/general-availability-of-azure-openai-service-expands-access-to-large-advanced-ai-models-with-added-enterprise-benefits/

[Piedone]

This service is for the same goal. We can check it out whether it makes sense to use it, or at least as an inspiration. I signed up for the waitlist.

[Piedone]

CodeRabbit does this, I'm testing it under Lombiq/Open-Source-Orchard-Core-Extensions#655.