Set a default user-agent #639
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This changes the user agent that is being sent to sites to match that of a browser. Some sites were blocking our requests because of the default user agent. node-fetch has some issues when attempting to set the user agent, so this was left alone for now, especially because in #552 there is a mention of potentially removing the user agent.
andreban
reviewed
Jan 13, 2022
PEConn
reviewed
Jan 27, 2022
| import nodefetch, {Response as NodeFetchResponse} from 'node-fetch'; | ||
|
|
||
| export type FetchEngine = 'node-fetch' | 'fetch-h2'; | ||
| const DEFAULT_FETCH_ENGINE: FetchEngine = 'fetch-h2'; | ||
|
|
||
| export type NodeFetchOrFetchH2Response = FetchH2Response | NodeFetchResponse; | ||
|
|
||
| const userAgent = 'Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/81.0'; | ||
| const fetchh2Ctx = context({userAgent: userAgent, overwriteUserAgent: true}); | ||
| const fetchh2 = fetchh2Ctx.fetch; |
There was a problem hiding this comment.
Are fetchh2Ctx and fetchh2 used anywhere?
There was a problem hiding this comment.
Yes, line 41 is where fetchh2 is used and fetchh2Ctx is used to generate the fetch call. Maybe we should collapse line 27 and 28 together?
There was a problem hiding this comment.
Oh right, you were previously importing something as fetchh2 but now you're creating a variable of the same name. Sorry I missed that
PEConn
approved these changes
Jan 28, 2022
| import nodefetch, {Response as NodeFetchResponse} from 'node-fetch'; | ||
|
|
||
| export type FetchEngine = 'node-fetch' | 'fetch-h2'; | ||
| const DEFAULT_FETCH_ENGINE: FetchEngine = 'fetch-h2'; | ||
|
|
||
| export type NodeFetchOrFetchH2Response = FetchH2Response | NodeFetchResponse; | ||
|
|
||
| const userAgent = 'Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/81.0'; | ||
| const fetchh2Ctx = context({userAgent: userAgent, overwriteUserAgent: true}); | ||
| const fetchh2 = fetchh2Ctx.fetch; |
There was a problem hiding this comment.
Oh right, you were previously importing something as fetchh2 but now you're creating a variable of the same name. Sorry I missed that
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This changes the user agent that is being sent to sites to match that of
a browser. Some sites were blocking our requests because of the default
user agent. node-fetch has some issues when attempting to set the user
agent, so this was left alone for now, especially because in #552 there
is a mention of potentially removing the user agent.