Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add client.SetMultipartBoundaryFunc and port Blink/WebKit/Firefox implementations #392

Merged
merged 1 commit into from
Oct 14, 2024

Conversation

rosahaj
Copy link
Contributor

@rosahaj rosahaj commented Oct 13, 2024

Problem

req currently relies on Go's mime/multipart package to generate unique boundary delimiters for multipart/form-data requests. The boundary delimiters generated by mime/multipart are made up of exactly 60 randomly-generated characters (numbers from 0-9 and lower-case letters from a-f only), e.g. b26dc610159ba676929ced239b36e90370976ad0fc20a615e29e799608d2 (see implementation).

All modern browsers use boundary delimiter formats different from those used by mime/multipart:

  • Blink-based browsers (Chrome, Chromium, Edge, etc.) and WebKit-based browsers (Safari, etc.) generate boundary delimiters in the format of ----WebKitFormBoundary[a-zA-Z0-9]{16}.
  • Firefox-based browsers generate boundary delimiters in the format of -------------------------\d{1,10}\d{1,10}\d{1,10}.

It's very easy for bot detection tools to detect whether the boundary delimiter used in a multipart/form-data request could have originated from the browser indicated by the request's User-Agent header, with a near-zero risk of false positives, and there's evidence that some websites have implemented such checks (e.g. Discord).

Solution

This PR introduces a new client.SetMultipartBoundaryFunc(fn func() string) method, which accepts a custom function that's called every time a new multipart/form-data request is sent.

Furthermore, it adds ports of the Blink/WebKit and Firefox implementations for boundary delimiter generation to client.ImpersonateChrome(), client.ImpersonateSafari() and client.ImpersonateFirefox().

It also includes test cases for client.SetMultipartBoundaryFunc and the Blink/WebKit and Firefox ports.

@imroc imroc merged commit 24b0c84 into imroc:master Oct 14, 2024
2 checks passed
@imroc
Copy link
Owner

imroc commented Oct 14, 2024

Nice feature, thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants