Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pass timeout to request, so it is propagated into StreamRead and affects .read method #86

Merged
merged 8 commits into from
Jan 13, 2017

Conversation

vharitonsky
Copy link
Contributor

Currently StreamReader returned by aiohttp is not affected by read timeout passed into client, so .read method has 300 seconds timeout by default, we should pass the read timeout into request so it is set on StreamReader

@thehesiod
Copy link
Collaborator

thehesiod commented Dec 30, 2016

looking through aiohttp this does seem to be the correct fix for read_timeout. Note: this ends up going to aiohttp.streams.StreamReader. Normally we'd want a unit test however to test this one would have to mock aiohttp and that infrastructure doesn't seem available yet in the existing aiobotocore tests. Asking @jettify to give final approval.

@thehesiod thehesiod mentioned this pull request Dec 30, 2016
@thehesiod
Copy link
Collaborator

btw, I think with this the asyncio.wait_for can be removed right? It should now be handled by the TCPConnector's conn_timeout (already set), and now the read_timeout set by this PR.

@thehesiod
Copy link
Collaborator

thehesiod commented Jan 3, 2017

btw, per my research mentioned in #88 the removal of the wait_for is indeed correct iff read_timeout is >= conn_timeout (default is 60s each). Could you please add an assert/warning and comment to ensure this is the case in the future...then I'll merge this given jettify hasn't responded yet.

@thehesiod thehesiod assigned thehesiod and unassigned jettify Jan 3, 2017
@thehesiod thehesiod self-requested a review January 3, 2017 20:57
@thehesiod thehesiod removed their assignment Jan 3, 2017
Copy link
Collaborator

@thehesiod thehesiod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add assert + comment or warning to ensure read_timeout >= conn_timeout since aiohttp adds 'timeout' (now read_timeout) wrapper around the TCPConnector connect. FYI I've created a PR for aiohttp to later be able to clearly set the read + connection timeouts: aio-libs/aiohttp#1523

@thehesiod
Copy link
Collaborator

I've gone ahead and added the warning to your branch.

- bump up version due to timeout handling change
- add changes
- fix flake
- add changes missing from
https://github.com/aio-libs/aiobotocore/pull/74/files
- fix url due to use of yarl by the latest aiohttp
@thehesiod
Copy link
Collaborator

thehesiod commented Jan 13, 2017

after running unittests I found that aiobotocore was broken for two reasons:

  1. https://github.com/aio-libs/aiobotocore/pull/74/files was incomplete
  2. latest aiohttp broken aiobotocore because it switched to using YARL and assumes that the incoming url is not encoded.

both have been fixed in this PR along with bumping up the aiohttp requirement

@thehesiod thehesiod merged commit 988b893 into aio-libs:master Jan 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants