Support of Accept-Encoding header

[talset]

For a few days we receive 403 on URL such https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token

curl -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0' https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token -I
HTTP/2 403

To fix the issue, we need to provide Accept-Encoding to blc

curl -H 'Accept-Encoding: gzip, deflate, br' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:99.0) Gecko/20100101 Firefox/99.0' https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token -I
HTTP/2 200

Describe the solution you'd like
Ideal it would be nice to be able to provide custom headers

Describe alternatives you've considered
Adding a flag such as --user-agent to provider --accept-encoding too

Additional context

[cnunciato]

It’s a bit of a hack — actually it’s a pretty big one 😂 — but if you find yourself in need a workaround, as we did, you can patch the underlying library:

const bhttp = require("bhttp");
const oldRequest = bhttp.request;
bhttp.request = function() {
    const [ url, options, callback ] = arguments;

    // Modify request options.
    // https://git.cryto.net/joepie91/node-bhttp/src/branch/master/lib/bhttp.js#L886
    options.headers.accept = "*/*";

    return oldRequest.apply(this, arguments);
};

https://github.com/pulumi/docs/blob/master/scripts/link-checker/check-links.js#L53