feat: add PullRequestDiffsStream as child to PullRequestsStream

[angelachenn]

This PR adds a child stream for pull request diffs to the parent pull requests stream. References Parent-Child Streams and Github REST API documentation.

• also adds pull_id to the PR stream parent context, making {org, repo, repo_id, pull_number, pull_id}. Gives users more flexibility in using or joining their tables.
• I followed code patterns that I noticed, except for matching error status codes against their respective messages; this is more to avoid a dependency on Github error messages. If they were to change them for whatever reason, that'd be a breaking change.

Some clarity on the error catching: all of them are related to the diff size being too large for to process and/or stream.

• 406, 422 and 502 are all from the Github REST API's end.
  • 406 being diff too large either because it exceeds the max # of files (300) or lines changed (20000)
  • 422 being server error because diff is taking too long to generate
  • 502 Server Error
• PR diffs that are small enough to call via the API but still quite large will result in this error:

Modifying the buffer size is risky and doesn't really prevent related issues from resurfacing downstream. Luckily intervening at the parsing step worked, I've set the content-size limit to 40MiB to allow for some overhead.

PRs that cause those errors are skipped. I've tested with a pipeline on my end and it all seems to be working successfully.

2025-01-23 08:16:29,000 | INFO     | tap-github.pull_request_diffs | Skipping PR due to 422 error: Server Error: Sorry, this diff is taking too long to generate.
2025-01-23 08:17:59,294 | INFO     | tap-github.pull_request_diffs | Skipping PR due to 406 error: Sorry, the diff exceeded the maximum number of files (300). Consider using 'List pull requests files' API or locally cloning the repository instead.
2025-01-23 08:43:26,939 | INFO     | tap-github.pull_request_diffs | Skipping PR. The diff size (55.72 MiB) exceeded the maximum size limit of 40 MiB.

My initial brute-force approach was just to shove another call to the diff in the post_process method within PullRequestsStreams based on a config setting, but that results in getting instantly rate-limited lol.

Please advise if I missed anything. Thanks!

[edgarrmondragon]

Thanks @angelachenn! Can you fix the linting issues? I'll review after that.

[angelachenn]

Thanks for the quick reply!! Looking into that right now

[edgarrmondragon]

pre-commit.ci autofix

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[edgarrmondragon]

Thanks @angelachenn!