test: AtCoderの鯖側の負荷が高いときにContent-Typeからcharset=utf-8が消える

[kmyk]

#13
AtCoderからサンプルケースを取得しようとすると、取得には成功するが 入力例 1 などの文字列が文字化けすることがある
コンテスト開始直後のみ半分ぐらいの割合で発生する

[kmyk]

現象が長めに続いていたので resp.headers と resp.encoding を出力してみた

$ ./oj dl -n https://beta.atcoder.jp/contests/abc085/tasks/abc085_a
[x] problem recognized: <onlinejudge.atcoder.AtCoderProblem object at 0x7f2388960940>: https://beta.atcoder.jp/contests/abc085/tasks/abc085_a
[*] load cookie from: /home/user/.local/share/onlinejudge/cookie.jar
[x] GET: http://abc085.contest.atcoder.jp/tasks/abc085_a
[x] 200 OK
{'Transfer-Encoding': 'chunked', 'X-ImoJudge-PassportCache': 'Cached', 'X-ImoJudge-PassportCacheKey': 'PassportCache-2-abc085.contest.atcoder.jp-/tasks/abc085_a', 'Content-Type': 'text/html', 'X-AtCoder-Cache': 'private', 'Content-Encoding': 'gzip', 'Set-Cookie': '_kick_id=2012-04-25+04%3A51; expires=Mon, 07-Jan-2019 12:08:02 GMT; Max-Age=31536000; path=/; domain=atcoder.jp', 'Date': 'Sun, 07 Jan 2018 12:08:02 GMT', 'X-ImoJudge-SimpleAuth': 'Passed', 'Expires': 'Fri, 1 Jan 2010 00:00:00 GMT', 'Server': 'nginx', 'Connection': 'keep-alive', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, no-store, must-revalidate, private'}
ISO-8859-1
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Input 1 
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Output 1 
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Input 2 
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Output 2 
[*] save cookie to: /home/user/.local/share/onlinejudge/cookie.jar

[*] sample 0
[x] input: å
¥å��ä¾� 1
2017/01/07
[x] output: ��� 1
2018/01/07

[*] sample 1
[x] input: å
¥å��ä¾� 2
2017/01/31
[x] output: ��� 2
2018/01/31

$ ./oj dl -n https://beta.atcoder.jp/contests/abc085/tasks/abc085_a
[x] problem recognized: <onlinejudge.atcoder.AtCoderProblem object at 0x7f830da64ac8>: https://beta.atcoder.jp/contests/abc085/tasks/abc085_a
[*] load cookie from: /home/user/.local/share/onlinejudge/cookie.jar
[x] GET: http://abc085.contest.atcoder.jp/tasks/abc085_a
[x] 200 OK
{'X-Twig-Cost': '0.0008sec', 'X-ImoJudgeAPI-Cost': '0.0640sec', 'Expires': 'Fri, 1 Jan 2010 00:00:00 GMT', 'Server': 'nginx', 'X-AtCoder-Cache': 'private', 'X-I18n-Cost': '0.0019sec', 'Set-Cookie': '_kick_id=2012-04-25+04%3A51; expires=Mon, 07-Jan-2019 12:08:04 GMT; Max-Age=31536000; path=/; domain=atcoder.jp, __privilege=contestant; expires=Mon, 07-Jan-2019 12:08:04 GMT; Max-Age=31536000; path=/; domain=abc085.contest.atcoder.jp', 'X-AtCoder-Cost': '0.0676sec', 'Cache-Control': 'no-cache, no-store, must-revalidate, private', 'X-AtCoder-User-ID': '10913', 'Pragma': 'no-cache', 'X-ImoJudge-PassportCache': 'Enabled', 'Date': 'Sun, 07 Jan 2018 12:08:04 GMT', 'X-AtCoder-IssueTime': '2018-01-07 21:08:04', 'Transfer-Encoding': 'chunked', 'Content-Encoding': 'gzip', 'X-ImoJudgeAPI-History': 'select/me,select/contests,select/me,select/assignments,select/tasks', 'X-AtCoder-Init-Cost': '0.0391sec', 'Content-Type': 'text/html; charset=utf-8', 'X-ImoJudge-SimpleAuth': 'Passed', 'X-ImoJudge-PassportCacheKey': 'PassportCache-2-abc085.contest.atcoder.jp-/tasks/abc085_a', 'Connection': 'keep-alive'}
utf-8
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Input 1 
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Output 1 
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Input 2 
[*] skipped due to language: current one is lang-ja, not lang-en: Sample Output 2 
[*] save cookie to: /home/user/.local/share/onlinejudge/cookie.jar

[*] sample 0
[x] input: 入力例 1
2017/01/07
[x] output: 出力例 1
2018/01/07

[*] sample 1
[x] input: 入力例 2
2017/01/31
[x] output: 出力例 2
2018/01/31

[kmyk]

cacheの関係とかだろうか
気にしなくてよさそう？ ISO-8859-1 になってたら適当にする処理入れる？