KAFKA-12359: Update Jetty to 11 #10176
Conversation
|
The first commit addresses the Java 11 Migration (KAFKA-12358) issue. If the community decides to migrate into Java 11, I will separate it into an independent PR. |
dongjinleekr
force-pushed
the
feature/KAFKA-12359
branch
from
ecc1d7a to
77ffb60
Compare
|
For those who are curious about how the Jetty 11 upgrade would be, here is the latest draft with Jetty 11.0.2. |
1. Change Java EE dependencies for Jetty 11 - javax.activation:activation:1.1.1 → com.sun.activation:javax.activation:1.2.0 - javax.ws.rs:javax.ws.rs-api 2.1.1 → jakarta.ws.rs:jakarta.ws.rs-api 3.0.0. - javax.xml.bind:jaxb-api:2.3.0 → jakarta.xml.bind:jakarta.xml.bind-api:2.3.0 - Change all javax.ws.rs.* imports into jakarta.ws.rs.*. 2. Upgrade jackson from 2.10.5 to 2.12.1 + Add jakarta classifier to jackson-jaxrs-json-provider for compatibility with jakarta.ws.rs-api - Update jackson-databind dependency: now it uses the same version with the other jackson dependencies. - Add '@JsonIgnoreProperties(ignoreUnknown = true)' annotation to ErrorResponse: Prevent 'com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "servlet"'. - Remove ScalaObjectMapper from ConsumerGroupCommand: deprecated from jackson-module-scala 2.12.1. - Mark Empty with @JsonSerialize, @JsonDeserialize: now serializable with Jetty 11. 3. Upgrade jersey from 2.31 to 3.0.1 4. Add additional dependencies for Jetty 11 - javax.ws.rs:jsr311-api:1.1.1 - jakarta.servlet:jakarta.servlet-api:5.0.0 - org.glassfish.jersey.media:jersey-media-json-jackson:3.0.1: Required to register jackson as a MessageBodyWriter. 5. Upgrade Jetty from 9.4.x to 11.0.2 - Exclude org.slf4j:slf4j-api:2.0.0-alpha1 dependency. - Fix deprecated method calls in RestClient#httpRequest, InternalRequestSignature#addToRequest - Update InternalRequestSignatureTest#addToRequestShouldThrowExceptionOnInvalidSignatureAlgorithm Since Request is an Interface from Jetty 11, mocking without when..then.. clause does not work. So it is now replaced to real instance. - Update SSLUtils From Jetty 11, SslContextFactory is separated into SslContextFactory.Server and SslContextFactory.Client subtypes; So, the return types of SSLUtils#[createServerSideSslContextFactory, createClientSideSslContextFactory] are now changed. Since SslContextFactory.Client does not have getNeedClientAuth, getWantClientAuth methods so SSLUtilsTest is also updated accordingly. - Increase JsonRestServer#GRACEFUL_SHUTDOWN_TIMEOUT_MS from 100 to 3000 Jetty 11 requires more time to tear down their resources than Jetty 9.4.x; without this modification, TimeoutException is thrown during shutdown in AgentTest, CoordinatorTest.
dongjinleekr
force-pushed
the
feature/KAFKA-12359
branch
from
77ffb60 to
3710267
Compare
|
We are not planning to upgrade to Java 11 any time soon. Can you please submit a PR for the latest Jetty version that supports Java 8 still? |
|
For those who are interested in this issue, here is some context: As you can see in the table below (from here), the change between Jetty 9.4 and Jetty 11 is not only Servlet versions; it involves Servlet version, package namespace, API changes, and related dependencies. For this reason, Upgrading Jetty from 9.4 is never a simple task but a complicated one that requires lots of changes, tests, etc.
The problem is: the current 9.4's End of Life seems not a too distant future; For instance, Jetty 9.2 reached the end of life in March 2018, and 9.3 also did in February 2020 (i.e., in two years). 9.3 also dropped Java 7 support from before its final release. Although the Jetty team does not mention it explicitly, the community seems to expect it to reach the EOL in 2022 when Jetty 10.x and 11.x becomes stable and widespread. If this thing happens, we should also migrate to Jetty 11, along with Java 11. I agree with @ijuma that it is too early to discuss this issue. However, I think we have enough reason to recognize this problem. I will keep my eyes on Jetty 9.4's EOL issue and as soon as it becomes certain, I will reopen the PR. +1. For those who hope to test Kafka Connect on Servlet 5: I will start providing a preview soon like log4j2 appender. |
|
Any update on this? Jetty 9.4 will not reach EOL until the last Webtide customer has migrated away from it (which may take forever more or less). It did however reach "End of community support" back in June 2022. Even Jetty 11 that this PR is migrating to has already reached "End of community support". References: |
|
I started a new thread on the dev list about this. I think it's very important to upgrade for Kafka 4.0 as many libraries have moved away from javax namespace. Would it be possible to re-open and update this PR? |
Change all
javax.ws.rs.*imports intojakarta.ws.rs.*.Make Jackson to be compatible with new Jakarta API.
MessageBodyWriter.Committer Checklist (excluded from commit message)