Why I am getting com.amazonaws.AbortedException & SdkInterruptedException while calling s3Client.getObject()

[DharmendraDubey]

We are getting following exceptions while calling s3Client.getObject(bucketName, key)

com.amazonaws.AbortedException:
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleInterruptedException(AmazonHttpClient.java:868)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:746)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5062)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5008)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1490)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1344)
at com.elliemae.core.security.nopool.S3CryptoClient.getEncByteArrayFromBucket(S3CryptoClient.java:245)
at com.elliemae.core.security.nopool.S3CryptoClient.getEncByteArrayFromBucket(S3CryptoClient.java:236)
at com.elliemae.core.security.nopool.SymmSecurityUtil.getEncryptedFromS3(SymmSecurityUtil.java:119)
at com.elliemae.audit.common.S3FileResource.getEncryptedFromS3(S3FileResource.java:54)
at com.elliemae.audit.mappings.ChangeSetProcessor.getLoanContent(ChangeSetProcessor.java:159)
at com.elliemae.audit.mappings.ChangeSetProcessor.processChangeSet(ChangeSetProcessor.java:83)
at com.elliemae.audit.common.LoanDiffEnricher$.transformLoanDiff(LoanDiffEnricher.scala:305)
at com.elliemae.audit.spark.streaming.AuditStreamer$.$anonfun$main$3(AuditStreamer.scala:315)
at scala.collection.Iterator$$anon$10.next(Iterator.scala:459)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:512)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)
at scala.collection.Iterator$SliceIterator.hasNext(Iterator.scala:266)
at scala.collection.Iterator.foreach(Iterator.scala:941)
at scala.collection.Iterator.foreach$(Iterator.scala:941)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
at scala.collection.generic.Growable.$plus$plus$eq(Growable.scala:62)
at scala.collection.generic.Growable.$plus$plus$eq$(Growable.scala:53)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:105)
at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:49)
at scala.collection.TraversableOnce.to(TraversableOnce.scala:315)
at scala.collection.TraversableOnce.to$(TraversableOnce.scala:313)
at scala.collection.AbstractIterator.to(Iterator.scala:1429)
at scala.collection.TraversableOnce.toBuffer(TraversableOnce.scala:307)
at scala.collection.TraversableOnce.toBuffer$(TraversableOnce.scala:307)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1429)
at scala.collection.TraversableOnce.toArray(TraversableOnce.scala:294)
at scala.collection.TraversableOnce.toArray$(TraversableOnce.scala:288)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1429)
at org.apache.spark.rdd.RDD.$anonfun$take$2(RDD.scala:1423)
at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2133)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:127)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:444)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:447)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)

2. Caused by: com.amazonaws.http.timers.client.SdkInterruptedException
   at com.amazonaws.http.AmazonHttpClient$RequestExecutor.checkInterrupted(AmazonHttpClient.java:923)
   at com.amazonaws.http.AmazonHttpClient$RequestExecutor.checkInterrupted(AmazonHttpClient.java:909)
   at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1103)
   at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
   at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
   at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
   ... 46 more

[debora-ito]

Can you share a sample code showing how you are using the SDK and that we can use to reproduce the error?
Which version of the SDK are you using?

[DharmendraDubey]

Hi Debora, we are using aws SDK version "1.11.490" in below sample code

public S3Object getS3OjectFromBucket(String bucketName, String key) {
AmazonS3 s3Client = null;
S3Object s3object = null;
try {
s3client = AmazonS3ClientBuilder.standard().withRegion(region)
      .withCredentials(awsCredentialsProvider).build();
s3object  = s3Client.getObject(bucketName, key); // getting the exception on this line
}
catch (Exception ex) {
// printing the exception here in logs
throw ex;
}
return s3Object;
}

[debora-ito]

A few suggestions of what can be causing the issue:

• Try to reuse one single instance of s3 client instead of creating a new one for every request
• Maybe something external to the code is closing/interrupting the connection
• version 1.11.490 is almost 4 years-old now, try using a newer version of the SDK