Skip to content
This repository has been archived by the owner on Dec 28, 2019. It is now read-only.

500 error on API calls (inconsistently -- API server flakiness) #62

Closed
IgnoredAmbience opened this issue Oct 25, 2019 · 7 comments
Closed
Assignees
Labels

Comments

@IgnoredAmbience
Copy link
Owner

Testcase: https://groups.yahoo.com/api/v1/groups/ColorComputer/messages/812/raw

Ensure handled gracefully.

@IgnoredAmbience IgnoredAmbience self-assigned this Oct 25, 2019
@IgnoredAmbience
Copy link
Owner Author

Confirmed handled gracefully now.

@IgnoredAmbience
Copy link
Owner Author

2019-10-25 23:06:59.699 BST INFO archive_message_content Fetching  raw message id: 812 (of 5091)
2019-10-25 23:06:59.892 BST ERROR archive_message_content Raw grab failed for message 812
Traceback (most recent call last):
  File "./yahoo.py", line 92, in archive_message_content
    raw_json = yga.messages(id, 'raw')
  File "/home/thomas/yahoo-group-archiver-1/yahoogroupsapi.py", line 110, in get_json
    raise e
  File "/home/thomas/yahoo-group-archiver-1/yahoogroupsapi.py", line 103, in get_json
    r.raise_for_status()
  File "/home/thomas/yahoo-group-archiver-1/env3/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://groups.yahoo.com/api/v1/groups/ColorComputer/messages/812/raw
2019-10-25 23:06:59.892 BST INFO archive_message_content Fetching html message id: 812 (of 5091)

@IgnoredAmbience
Copy link
Owner Author

I'm seeing 500 errors that are not persistent, seems to be API server flakiness. We definitely need a (deferred) retry mechanism here. And ideally tooling to pull failures out of already captured archive logs.

@IgnoredAmbience IgnoredAmbience changed the title 500 error on raw message 500 error on raw message (inconsistently -- API server flakiness) Oct 29, 2019
@IgnoredAmbience
Copy link
Owner Author

IgnoredAmbience commented Oct 29, 2019

For a transient error:

"ygError":{"hostname":"gapi5.grp.bf1.yahoo.com","httpStatus":500,"errorMessage":"Internal error: Error during message fetch","errorCode":1001,"sid":"SID:YHOO:groups.yahoo.com:ee5ae7585faace373012569deb353069:0"}}

@IgnoredAmbience
Copy link
Owner Author

Unfortunately the same error results for the raw message that always errors:

"ygError":{"hostname":"gapi1.grp.bf1.yahoo.com","httpStatus":500,"errorMessage":"Internal error: Error during message fetch","errorCode":1001,"sid":"SID:YHOO:groups.yahoo.com:4470210ff4f93c3f0d8ca3aa5cc5129a:0"}}

@IgnoredAmbience
Copy link
Owner Author

Additional retries added in 50eaa48

@d235j
Copy link
Collaborator

d235j commented Oct 30, 2019

This needs to be handled for any API call — suggest moving the logic to the YahooGroupsAPI module.

@IgnoredAmbience IgnoredAmbience changed the title 500 error on raw message (inconsistently -- API server flakiness) 500 error on API calls (inconsistently -- API server flakiness) Oct 30, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants