Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 uploads are 2 - 3 times slower in IOS compared to Android. #2434

Closed
chandusam2002 opened this issue Apr 15, 2020 · 18 comments
Closed

S3 uploads are 2 - 3 times slower in IOS compared to Android. #2434

chandusam2002 opened this issue Apr 15, 2020 · 18 comments
Labels
pending-community-response Issue is pending response from the issue requestor s3 Issues related to S3

Comments

@chandusam2002
Copy link

chandusam2002 commented Apr 15, 2020

Describe the bug
IOS S3 uploads(using TransferUtility/S3 Client) are slower compared to Android. Upload with size 8 MB takes anywhere between 20 and 30 seconds(on a wifi connection) where as the same upload is taking 7 - 9 seconds on Android.
3 MB upload is taking between 15 and 20 seconds.
I understand that IOS is different and uses NSURLSession to submit uploads. But the performance difference is too big.

** Additional Details**
Downloads are taking slightly longer on IOS but the difference is smaller compared to IOS. This behavior is observed over a long period of time over difference devices.
We use https://github.com/aws-amplify/aws-sdk-android for Android.

Which AWS service(s) are affected?
S3 uploads using either TransferUtility or S3 Client.

Expected behavior
Upload performance is similar to Android.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment(please complete the following information):

  • SDK Version: 2.12.2
  • Dependency Manager: CocoaPads
  • Swift Version : 4.2

Device Information (please complete the following information):

  • Device: iPhone 8 (both phone and simulator)
  • iOS Version: 12.1
  • Specific to simulators: No

Additional context
Tried using multipart uploads and S3 client and there is very little difference in the overall performance.
On Android side using a ByteArrayInputStream instead of writing to a temporary file has improved the performance. I see that IOS writes the Data object to a temporary file. Not sure whether that is causing the increase in latency.

@lawmicha
Copy link
Contributor

Hi @chandusam2002 interesting testing on performance using different clients. I was going to suggest trying to use S3 client directly to see if that improves the performance but it sounds like there is very little difference. The S3 client is the thinnest client we have to interact with S3 service so if the round trip time of the request isn't improved, there isn't much the SDK can improve on and would be something we can take as an enhancement to perhaps improve the AWSCore which contains a lot of the networking classes that are used across all SDKs.

i've heard there is this feature called "transfer acceleration", I'm not familar with it but sounds like worth a look https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html

@lawmicha lawmicha added s3 Issues related to S3 pending-community-response Issue is pending response from the issue requestor labels Apr 17, 2020
@chandusam2002
Copy link
Author

Thanks @lawmicha . I will run tests with transfer acceleration to see whether it improves performance.

@kylelin47
Copy link

kylelin47 commented Apr 27, 2020

I'm seeing the same issue. A simple PutObject for an 800 KB file will take ~2s using the Android SDK and around ~5s using this SDK.

I am using the API in the Android SDK that takes in a File however, while the iOS SDK does not appear to have an equivalent. Maybe there's something going on in the Android SDK that makes the file streaming more efficient, or something strange about how the content field is being read in the iOS SDK.

@kylelin47
Copy link

kylelin47 commented Apr 28, 2020

Seems more likely that it's due to how the SDKs connect to S3.

This SDK uses the deprecated https://s3.amazonaws.com/{bucket}/{key} URI and the PUT HTTP verb, while the Android AWS SDK uses the more up-to-date https://{bucket}.s3.amazonaws.com/{key} URI and may have some optimizations around how it connects.

What's interesting is that their performance is similar when I connect to a us-west-2 bucket (I am in Washington), but if I connect to a us-east-1 bucket Android far outstrips iOS.

@drochetti drochetti added follow up Requires follow up from maintainers and removed pending-community-response Issue is pending response from the issue requestor labels Oct 20, 2020
@rahulvyas
Copy link

I am also facing the same issue. Is this issue fixed ?

@kylelin47
Copy link

kylelin47 commented Mar 3, 2021

Is this related to sessions being configured as background sessions? https://github.com/aws-amplify/aws-sdk-ios/blob/main/AWSS3/AWSS3TransferUtility.m#L441

Is there a way to configure this to be foreground sessions when appropriate?

Though I suppose this only applies to TransferUtility, whereas this issue seems to exist for bare S3 clients as well. I also only see a way to configure transfer acceleration while using TransferUtility, and not on the bare S3 client, which means I'd maybe be taking a step backward if I wanted to use the S3 client for its foregrounding.

So I suppose I'd like one of:

  1. The ability to use transfer acceleration on the S3 client
  2. The ability to make TransferUtility act as a foreground uploader/downloader

@palpatim
Copy link
Member

palpatim commented Mar 3, 2021

In the intervening time since this issue was opened, we updated the S3 SDK to use virtual host-style S3 endpoints for the S3 client. It would be interesting to hear if you are still seeing a performance difference using direct S3 client access.

However, as @kylelin47 points out, the AWSS3TransferUtility hands off all transfers to a background NSURLSession, which may have some impact on performance since iOS can manage the priority of background operations. In addition, the TransferUtility already used virtual-host style URLs even before the SDK change, so I would not expect the virtual host change to impact TransferUtility.

@palpatim palpatim added pending-community-response Issue is pending response from the issue requestor and removed follow up Requires follow up from maintainers labels Mar 3, 2021
@kylelin47
Copy link

kylelin47 commented Mar 4, 2021

That's good to know. I would like to use the S3 client directly, but last I checked I was not able to enable the use of transfer-accelerated calls. Has this been added to the SDK/always existed and I missed it?

@palpatim
Copy link
Member

palpatim commented Mar 5, 2021

That's good to know. I would like to use the S3 client directly, but last I checked I was not able to enable the use of transfer-accelerated calls. Has this been added to the SDK/always existed and I missed it?

It probably wouldn't have worked before the change to support VHost style URLs, but I have verified that transfer acceleration does work using the S3 SDK. See #3460 for a newly-added integration test that demonstrates it.

Hope this helps.

@kylelin47
Copy link

🔥 I'll try it out!

@royjit
Copy link
Contributor

royjit commented May 18, 2021

Closing this ticket due to no activity.

@royjit royjit closed this as completed May 18, 2021
@maanvendersingh89
Copy link

Hey guys,

Did you find any solutions for this issue. Please let me know so I can use same solution.
IOS 14
AWS 2.19.1

@harshvk4
Copy link

Same with 2.26.1 and iOS 15

@sspogra
Copy link

sspogra commented Dec 1, 2021

Same with 2.26.5 and iOS 15.

@VivekDhole9414
Copy link

VivekDhole9414 commented Apr 15, 2022

on iOS 14 same image uploading within second but on iOS 15.4 same image take a too much time to upload with same code
please help me for iOS 15.4 image ,video and other type of media file take too much time to upload with SDK version is 2.26.6

@EugeneNaloiko
Copy link

Same in 2023. The upload speed is not consistent. Sometimes it's quite ok, sometimes 2-3 times slower.

@atierian
Copy link
Member

@EugeneNaloiko which version of AWS SDK iOS are you using? This was fixed with 2.31.1.

@EugeneNaloiko
Copy link

@EugeneNaloiko which version of AWS SDK iOS are you using? This was fixed with 2.31.1.

2.19.1. Hmm, strange, didn't define the exact version, but it's not getting updated to latest automatically on pod update. Thanks, will try out newer one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pending-community-response Issue is pending response from the issue requestor s3 Issues related to S3
Projects
None yet
Development

No branches or pull requests