Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrading to 0.29.0-Beta introduces slf4j dependency hell: NoSuchMethodError #993

Closed
mgroth0 opened this issue Jul 22, 2023 · 10 comments · Fixed by smithy-lang/smithy-kotlin#963
Labels
bug This issue is a bug.

Comments

@mgroth0
Copy link

mgroth0 commented Jul 22, 2023

Describe the bug

I'm using the sdk in gradle. That is, in a gradle "plugin" that shares a classpath with the Gradle API. Things were fine before, but suddenly after the new update I'm getting a NoSuchMethodError.

Expected behavior

No error

Current behavior

On an sdk method call, I get the following:

java.lang.NoSuchMethodError: 'boolean org.slf4j.Logger.isEnabledForLevel(org.slf4j.event.Level)'
	at aws.smithy.kotlin.runtime.telemetry.logging.slf4j.Slf4JLoggerAdapter.isEnabledFor(Slf4jLoggerProvider.kt:38)
	at aws.smithy.kotlin.runtime.telemetry.logging.CoroutineContextLogExtKt.log(CoroutineContextLogExt.kt:74)
	at aws.smithy.kotlin.runtime.http.operation.OperationHandler.call(SdkOperationExecution.kt:398)
	at aws.smithy.kotlin.runtime.http.operation.OperationHandler.call(SdkOperationExecution.kt:200)
	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt$execute$$inlined$withSpan$1.invokeSuspend(CoroutineContextTraceExt.kt:126)
	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt$execute$$inlined$withSpan$1.invoke(CoroutineContextTraceExt.kt)
	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt$execute$$inlined$withSpan$1.invoke(CoroutineContextTraceExt.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:167)
	at kotlinx.coroutines.BuildersKt.withContext(Unknown Source)
	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt.execute(SdkHttpOperation.kt:157)
	at aws.sdk.kotlin.services.s3.DefaultS3Client.getObject(DefaultS3Client.kt:2169)

Steps to Reproduce

Creating a perfect reproducer seems very time consuming since it involves getting all the dependencies and build logic right. But essentially, it might be something like:

  1. Create a library distribution (folder of jars) that uses the SDK with version 0.29.0-Beta
  2. In settings.gradle.kts, load this folder of jars like so:
buildscript {
    dependencies {
        compile(fileTree(jarFolder))
    }
}
  1. During gradle configuration, try to use the SDK.
  2. You might observe the error

Possible Solution

I'm not sure if the reproduction steps above would work, because its a sort of complex environment that I have and something might be missing. But in any case, I was able to figure out a few things while I was trying to debug this.

First of all, I added this to my code immediately before calling the sdk:

kotlin

println("Logger class source = ${org.slf4j.Logger::class.java.protectionDomain.codeSource.location}")

The output of that is:

Logger class source = file:/Users/matthewgroth/.gradle/wrapper/dists/gradle-8.2.1-all/d8pvvlun5bx6sdtwqhf8y9z4b/gradle-8.2.1/lib/slf4j-api-1.7.30.jar

This is interesting. Note the following:

  • aws.smithy.kotlin:logging-slf4j2-jvm:0.23.0 has a runtime dependency on slf4j-api 2.0.6 (see: https://repo1.maven.org/maven2/aws/smithy/kotlin/logging-slf4j2-jvm/0.23.0/logging-slf4j2-jvm-0.23.0.pom)
  • I made absolute sure that an updated slf4j-api jar was included on the class path. slf4j-api-2.0.6 is in fact added as a compile dependency in my settings.gradle.kts right next to the sdk.
  • The method in question isEnabledForLevel is in fact only present in slf4j 2.x.
  • I even tried adding slf4j-nop to the classpath, but this did not help.
  • I'm guessing that gradle loads up the Logger class from its own internal slf4j 1.x jar before I even add the sdk.

At a bare minimum, having a way to disable logging completely to dodge this error in the meantime while a more robuse solution is eventually implemented would be great.

Context

Gradle Version: 8.2.1
Kotlin Version: 1.9.0

I was in sort of desperate need for the new update, because I kept running into #905

The above issue does have a workaround. I have implemented custom retry logic, which mostly works. But even this is failing sometimes. I'm very eager to try the solution that was seemingly implemented in 0.29.0

When I downgrade back to 0.28.0-beta, I dodge the slf4j dependency hell issue, but re-introduce the old java.io.EOFException: \n not found: limit=0 content=… from the above referenced issue.

AWS Kotlin SDK version used

0.29.0-beta

Platform (JVM/JS/Native)

JVM

Operating System and version

Mac

@mgroth0 mgroth0 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 22, 2023
@mgroth0
Copy link
Author

mgroth0 commented Jul 22, 2023

I am testing 0.28.2-beta right now. So far no errors. I will update this if I see any.

@aajtodd
Copy link
Contributor

aajtodd commented Jul 24, 2023

Thanks for the issue. Indeed in 0.29.0-beta we took a dependency on SLF4J 2.x whereas previously we were on 1.x.

  1. What are you trying to do/create?
  2. It's not clear what the setup is here: load this folder of jars like so:, how are you managing dependencies?
  3. Are you dependent on a specific SLF4J version or you just happen to use whatever the SDK does and there is a conflict now with gradle?

During gradle configuration, try to use the SDK.

  1. From this comment it would appear as if you are trying to use the SDK from a gradle script (and if I'm reading this right during the configure phase and not while executing a task).

I can provide some suggestions perhaps but it doesn't really look like an issue with the SDK, it's an issue with your classpath + gradle.

@aajtodd aajtodd added response-requested Waiting on additional info and feedback. Will move to 'closing-soon' in 5 days. and removed needs-triage This issue or PR still needs to be triaged. labels Jul 24, 2023
@mgroth0
Copy link
Author

mgroth0 commented Jul 24, 2023

@aajtodd Thanks for the response and for the offer to provide some advice.

Sorry my original issue was so messy. I did not even want to try to make a reproducer because my project was so complex. But I did actually just try now, and luckily I was wrong. It was actually very simple and easy to reproduce!

https://github.com/mgroth0/aws-slf4j-dep-hell

There it is. If you clone and try to run any gradle command there, you should hopefully see the error.

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to 'closing-soon' in 5 days. label Jul 24, 2023
@sdhuka
Copy link

sdhuka commented Aug 14, 2023

@aajtodd I am running into the same issue after upgrading to 0.29.1-beta. We are not explicitly using any slf4j version in our project.

2023-08-14 16:31:37.097 AndroidRuntime                      com.amplifyframework.datastore.test            E  FATAL EXCEPTION: DefaultDispatcher-worker-1
                                                                                                              Process: com.amplifyframework.datastore.test, PID: 32067
                                                                                                              java.lang.NoSuchMethodError: No interface method isEnabledForLevel(Lorg/slf4j/event/Level;)Z in class Lorg/slf4j/Logger; or its super classes (declaration of 'org.slf4j.Logger' appears in /data/app/~~1sFp6aV0X1Tvgxt4PSQpwQ==/com.amplifyframework.datastore.test-LdaZAqRnuHSbfqk5_vkvCA==/base.apk!classes25.dex)
                                                                                                              	at aws.smithy.kotlin.runtime.telemetry.logging.slf4j.Slf4JLoggerAdapter.isEnabledFor(Slf4jLoggerProvider.kt:38)
                                                                                                              	at aws.smithy.kotlin.runtime.telemetry.logging.CoroutineContextLogExtKt.log(CoroutineContextLogExt.kt:74)
                                                                                                              	at aws.smithy.kotlin.runtime.http.operation.OperationHandler.call(SdkOperationExecution.kt:399)
                                                                                                              	at aws.smithy.kotlin.runtime.http.operation.OperationHandler.call(SdkOperationExecution.kt:200)
                                                                                                              	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt$execute$$inlined$withSpan$1.invokeSuspend(CoroutineContextTraceExt.kt:126)
                                                                                                              	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt$execute$$inlined$withSpan$1.invoke(Unknown Source:8)
                                                                                                              	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt$execute$$inlined$withSpan$1.invoke(Unknown Source:4)
                                                                                                              	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
                                                                                                              	at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:167)
                                                                                                              	at kotlinx.coroutines.BuildersKt.withContext(Unknown Source:1)
                                                                                                              	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt.execute(SdkHttpOperation.kt:157)
                                                                                                              	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt.roundTrip(SdkHttpOperation.kt:84)
                                                                                                              	at aws.sdk.kotlin.services.cognitoidentity.DefaultCognitoIdentityClient.getId(DefaultCognitoIdentityClient.kt:319)
                                                                                                              	at com.amplifyframework.auth.cognito.actions.FetchAuthSessionCognitoActions$fetchIdentityAction$$inlined$invoke$1.execute(Action.kt:74)
                                                                                                              	at com.amplifyframework.statemachine.ConcurrentEffectExecutor$execute$1$1.invokeSuspend(ConcurrentEffectExecutor.kt:26)
                                                                                                              	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
                                                                                                              	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
                                                                                                              	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
                                                                                                              	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
                                                                                                              	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
                                                                                                              	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
                                                                                                              	Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@16f5050, Dispatchers.Default]

@aajtodd
Copy link
Contributor

aajtodd commented Aug 15, 2023

@sdhuka somewhere in your dependency closure slf4j 1.x is being pulled in. Your options are:

  1. Figure out which dependency is introducing it and see if there is already a solution (e.g. newer version that pulls in 2.x) or explicitly deny slf4j 1.x in favor of 2.x (see https://docs.gradle.org/current/userguide/dependency_downgrade_and_exclude.html). IIRC 2.x should be binary compatible.
  2. We can introduce an slf4j 1.x version of the logger implementation. By default it's still going to depend on 2.x but you can use the gradle dependency management APIs linked above to override and substitute one for the other.

(1) is maybe the quickest route (assuming there is a solution). We plan on doing (2) anyway at some point but it wasn't on our roadmap yet.

@canatella
Copy link

It seems gradle it self is introducing the dependency ? gradle/gradle#26348

We're hit with #905 because we upload a jar for signing to s3 using the sdk and know we are stuck. I can't update to >= 0.29 until gradle fixes the issue on their side which will take time they say.

Any idea on how to move forward?

@aajtodd
Copy link
Contributor

aajtodd commented Sep 20, 2023

If you can't figure out a way to correct your classpath to use SLF4J 2.x then you might be able to try excluding the slf4j2 implementation that the default telemetry provider uses and then supplying an implementation based on slf4j 1.x

// file: build.gradle.kts
dependencies {
    implementation("aws.sdk.kotlin:s3:<version>) {
        exclude(group = "aws.smithy.kotlin", module = "logging-slf4j2")
    }
}

This will require you provide a binding though at the expected location used by the default provider.

e.g. You would need to implement a corresponding type using the SLF4j 1.x API at the same import path:

// file: your custom 1.x implementation file

package aws.smithy.kotlin.runtime.telemetry.logging.slf4j

/**
 * SLF4J 1.x based logger provider
 */
public object Slf4jLoggerProvider : LoggerProvider {
    override fun getOrCreateLogger(name: String): Logger {
        val sl4fjLogger = LoggerFactory.getLogger(name)
        return Slf4JLoggerAdapter(sl4fjLogger)
    }
}

private class Slf4jLoggerAdapter(private val delegate: org.slf4j.Logger) : Logger {
    // your 1.x adapter code
}

Caveat I haven't tested the above but thats the general idea. You strip the SLF4J 2.x implementation and provide a 1.x equivalent at the same import location. JVM class loader would then pickup your implementation at runtime. We may provide a 1.x binding directly one day but the default will remain 2.x so you would still have to do some work in your build script to exclude dependencies and swap them.

@NatanLifshitz
Copy link

If anyone has a working example of the above I would love to see it.

@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@aajtodd
Copy link
Contributor

aajtodd commented Oct 2, 2023

This should be fixed starting in 0.32.3-beta. If you continue to encounter issues please open a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants