Skip to content

Commit

Permalink
Fix JSR305 links yet again.
Browse files Browse the repository at this point in the history
I broke them when I added links to the Checker Framework, which began to claim javax.annotations, just as the JDK had previously done.

I had hoped that the fix was as simple as moving the Checker Framework to the -linkoffline section so that it could come after JSR305, but that didn't work: Maven tries to prevalidate the package-list file, and somehow it fails to connect to checkerframework.org.

We saw problems with pre-validation before with JSR305 in []But I diagnosed it wrong there: I thought that Maven was trying to fetch the Javadoc root URL -- e.g., https://checkerframework.org/api/. But I was wrong: I was confusing the `link` and `linkUri` variables in the maven-javadoc-plugin source. In fact, Maven tries to fetch the package-list file, as you'd expect. But for some reason, this fails for javadoc.io and for checkerframework.org, even though it succeeds for docs.oracle.com. But it doesn't appear to be a User-Agent problem: I'm able to wget from checkerframework.org with the default Java User-Agent (which fails in the javadoc.io case), an absent User-Agent, and the User-Agent that I think maven-javadoc-plugin is using (judging from its source code). Plus, as far as I can tell, the connection attempt doesn't even get far enough to send a User-Agent.

Here are the logs provided by `mvn javadoc:javadoc -Dorg.slf4j.simpleLogger.log.org.apache.http=debug`. (Note: Even *that* logging may go away with the upgrade to maven-javadoc-plugin 3.) Both javadoc.io and checkerframework.org follow the same pattern:

[DEBUG] Connection request: [route: {s}->https://checkerframework.org][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
[DEBUG] Connection leased: [id: 1][route: {s}->https://checkerframework.org][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
[DEBUG] Connecting to checkerframework.org:443
[DEBUG] Connection org.apache.http.impl.conn.DefaultClientConnection@45545e7a closed
[DEBUG] Connection org.apache.http.impl.conn.DefaultClientConnection@45545e7a shut down
[DEBUG] Connection org.apache.http.impl.conn.DefaultClientConnection@45545e7a closed
[DEBUG] Connection released: [id: 1][route: {s}->https://checkerframework.org][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
[DEBUG] Connection manager is shutting down
[DEBUG] Connection manager shut down
[ERROR] Error fetching link: https://checkerframework.org/api//package-list. Ignored it.

Compare to the successful logs for docs.oracle.com:

[DEBUG] Connection request: [route: {s}->https://docs.oracle.com][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
[DEBUG] Connection leased: [id: 0][route: {s}->https://docs.oracle.com][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
[DEBUG] Connecting to docs.oracle.com:443
[DEBUG] CookieSpec selected: best-match
[DEBUG] Auth cache not set in the context
[DEBUG] Target auth state: UNCHALLENGED
[DEBUG] Proxy auth state: UNCHALLENGED
[DEBUG] Attempt 1 to execute request
[DEBUG] Sending request: GET /javase/9/docs/api//package-list HTTP/1.1
...

So what is wrong? Probably SSL. Based on that suspicion, I learned that I could set -Djavax.net.debug=ssl, which produced more information before the close message:

main, WRITE: TLSv1.2 Handshake, length = 239
main, READ: TLSv1 Alert, length = 2
main, RECV TLSv1.2 ALERT:  fatal, handshake_failure
main, called closeSocket()
main, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
main, IOException in getSession():  javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
main, called close()
main, called closeInternal(true)
[DEBUG] Connection org.apache.http.impl.conn.DefaultClientConnection@3667faa8 closed

And now that I've written that, I can't reproduce it :\ With -Djavax.net.debug=*all*, I can see what else is going on:

%% Cached client session: [Session-1, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384]
main, called close()
main, called closeInternal(true)
main, SEND TLSv1.2 ALERT:  warning, description = close_notify
main, WRITE: TLSv1.2 Alert, length = 26
main, called closeSocket(true)
[DEBUG] Connection org.apache.http.impl.conn.DefaultClientConnection@58af5076 closed

Maybe the server has an aggressive timeout? I think I saw someone suggest that as a possibility.

I don't know more detail than that. I see people who suggest that the problem could involve:
- connection managers (which maven-javadoc-plugin uses): https://stackoverflow.com/a/24084141/28465
- a need for a specific key store: https://stackoverflow.com/a/5206837/28465
- a need for extra crypto implementations/settings: https://stackoverflow.com/a/38264878/28465
- or other settings (I at least tried passing -Dhttps.protocols= and adding the suggested (possibly wrong in this case) -Dhttps.cipherSuites to that): https://stackoverflow.com/a/32226688/28465
- a new JDK: https://stackoverflow.com/a/39405982/28465
- a new HTTPClient (though whatever comes with the newest maven-javadoc-plugin (assuming that that's built in to the plugin and not to Maven??) apparently wasn't good enough): https://stackoverflow.com/a/43220038/28465
- aggressive timeout, I think, as noted above
- maybe weird things happen when maven-javadoc-plugin closes the stream without reading from it?
- I think other things that I've forgotten, and surely other things I haven't understood.

Anyway, I made a local copy of the package-list, just as I'd done for the javadoc.io projects, even though this time there's only one problem instead of two :)

I'm pretty sure that the Checker Framework links will work now, too, but I haven't tested because we don't have any yet in Guava itself.

Possibly I should just switch us to back using -link instead of -linkoffline, adding the User-Agent override that I figured out last time....

Also, I moved some configuration to the parent pom so that it also affects guava-testlib. But I didn't move the most interesting parts, which require a little more work (though hopefully not much...).

Re-fixes #2965

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178252119
  • Loading branch information
cpovirk committed Dec 7, 2017
1 parent 9df5d9e commit 76f93b3
Show file tree
Hide file tree
Showing 6 changed files with 242 additions and 22 deletions.
101 changes: 101 additions & 0 deletions android/guava/javadoc-link/checker-framework/package-list
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
android.annotation
android.support.annotation
com.sun.istack.internal
edu.umd.cs.findbugs.annotations
javax.annotation
javax.annotation.concurrent
javax.annotation.meta
javax.validation.constraints
lombok
net.jcip.annotations
org.checkerframework.checker.compilermsgs
org.checkerframework.checker.compilermsgs.qual
org.checkerframework.checker.fenum
org.checkerframework.checker.fenum.qual
org.checkerframework.checker.formatter
org.checkerframework.checker.formatter.qual
org.checkerframework.checker.guieffect
org.checkerframework.checker.guieffect.qual
org.checkerframework.checker.i18n
org.checkerframework.checker.i18n.qual
org.checkerframework.checker.i18nformatter
org.checkerframework.checker.i18nformatter.qual
org.checkerframework.checker.i18nformatter.unittests
org.checkerframework.checker.index
org.checkerframework.checker.index.lowerbound
org.checkerframework.checker.index.qual
org.checkerframework.checker.index.samelen
org.checkerframework.checker.index.searchindex
org.checkerframework.checker.index.substringindex
org.checkerframework.checker.index.upperbound
org.checkerframework.checker.initialization
org.checkerframework.checker.initialization.qual
org.checkerframework.checker.interning
org.checkerframework.checker.interning.qual
org.checkerframework.checker.linear
org.checkerframework.checker.linear.qual
org.checkerframework.checker.lock
org.checkerframework.checker.lock.qual
org.checkerframework.checker.nullness
org.checkerframework.checker.nullness.compatqual
org.checkerframework.checker.nullness.qual
org.checkerframework.checker.propkey
org.checkerframework.checker.propkey.qual
org.checkerframework.checker.regex
org.checkerframework.checker.regex.qual
org.checkerframework.checker.signature
org.checkerframework.checker.signature.qual
org.checkerframework.checker.signedness
org.checkerframework.checker.signedness.qual
org.checkerframework.checker.tainting
org.checkerframework.checker.tainting.qual
org.checkerframework.checker.units
org.checkerframework.checker.units.qual
org.checkerframework.common.aliasing
org.checkerframework.common.aliasing.qual
org.checkerframework.common.basetype
org.checkerframework.common.reflection
org.checkerframework.common.reflection.qual
org.checkerframework.common.subtyping
org.checkerframework.common.util
org.checkerframework.common.util.count
org.checkerframework.common.util.debug
org.checkerframework.common.util.report
org.checkerframework.common.util.report.qual
org.checkerframework.common.value
org.checkerframework.common.value.qual
org.checkerframework.common.value.util
org.checkerframework.common.wholeprograminference
org.checkerframework.dataflow.analysis
org.checkerframework.dataflow.cfg
org.checkerframework.dataflow.cfg.block
org.checkerframework.dataflow.cfg.node
org.checkerframework.dataflow.cfg.playground
org.checkerframework.dataflow.constantpropagation
org.checkerframework.dataflow.qual
org.checkerframework.dataflow.util
org.checkerframework.framework.flow
org.checkerframework.framework.qual
org.checkerframework.framework.source
org.checkerframework.framework.test
org.checkerframework.framework.test.diagnostics
org.checkerframework.framework.type
org.checkerframework.framework.type.treeannotator
org.checkerframework.framework.type.typeannotator
org.checkerframework.framework.type.visitor
org.checkerframework.framework.util
org.checkerframework.framework.util.defaults
org.checkerframework.framework.util.dependenttypes
org.checkerframework.framework.util.element
org.checkerframework.framework.util.typeinference
org.checkerframework.framework.util.typeinference.constraint
org.checkerframework.framework.util.typeinference.solver
org.checkerframework.javacutil
org.checkerframework.javacutil.dist
org.checkerframework.javacutil.trees
org.eclipse.jdt.annotation
org.eclipse.jgit.annotations
org.jetbrains.annotations
org.jmlspecs.annotation
org.netbeans.api.annotations.common
org.springframework.lang
25 changes: 14 additions & 11 deletions android/guava/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -108,16 +108,19 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<configuration>
<encoding>UTF-8</encoding>
<docencoding>UTF-8</docencoding>
<charset>UTF-8</charset>
<additionalparam>-XDignore.symbol.file</additionalparam>
<additionalparam>-Xdoclint:-html</additionalparam>
<!-- TODO(cpovirk): Move this to the parent after making jdk-sources available there. -->
<!-- TODO(cpovirk): can we use includeDependencySources and a local com.oracle.java:jdk-lib:noversion:sources instead of all this unzipping and manual sourcepath modification? -->
<!-- (We need JDK *sources*, not just -link, so that {@inheritDoc} works.) -->
<sourcepath>${project.build.sourceDirectory}:${project.build.directory}/jdk-sources</sourcepath>

<subpackages>com.google.common</subpackages>
<excludePackageNames>com.google.common.base.internal</excludePackageNames>
<linksource>true</linksource>

<!-- TODO(cpovirk): Move this to the parent after making the package-list files available there. -->
<!-- We add the link ourselves, both so that we can choose Java 9 over the version that -source suggests and so that we can solve the JSR305 problem described below. -->
<detectJavaApiLink>false</detectJavaApiLink>
<offlineLinks>
<!-- Even after we stop using JSR305 annotations in our own code, we'll want this link so that NullPointerTester's docs can link to @CheckForNull and friends... at least once we start using this config for guava-testlib. -->
<offlineLink>
<url>https://static.javadoc.io/com.google.code.findbugs/jsr305/3.0.1/</url>
<location>${project.basedir}/javadoc-link/jsr305</location>
Expand All @@ -131,15 +134,15 @@
<url>https://docs.oracle.com/javase/9/docs/api/</url>
<location>https://docs.oracle.com/javase/9/docs/api/</location>
</offlineLink>
<!-- The Checker Framework likewise would claim javax.annotations, despite providing only a subset of the JSR305 annotations, so it must likewise come after JSR305. -->
<offlineLink>
<url>https://checkerframework.org/api/</url>
<location>${project.basedir}/javadoc-link/checker-framework</location>
</offlineLink>
</offlineLinks>
<links>
<link>http://errorprone.info/api/latest/</link>
<link>https://checkerframework.org/api/</link>
</links>
<!-- TODO(cpovirk): can we use includeDependencySources and a local com.oracle.java:jdk-lib:noversion:sources instead of all this unzipping and manual sourcepath modification? -->
<!-- (We need JDK *sources*, not just -link, so that {@inheritDoc} works.) -->
<sourcepath>${project.build.sourceDirectory}:${project.build.directory}/jdk-sources</sourcepath>
<subpackages>com.google.common</subpackages>
</configuration>
<executions>
<execution>
Expand Down
6 changes: 6 additions & 0 deletions android/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,12 @@
<configuration>
<notimestamp>true</notimestamp>
<additionalparam>-Xdoclint:-html</additionalparam>
<encoding>UTF-8</encoding>
<docencoding>UTF-8</docencoding>
<charset>UTF-8</charset>
<additionalparam>-XDignore.symbol.file</additionalparam>
<additionalparam>-Xdoclint:-html</additionalparam>
<linksource>true</linksource>
</configuration>
<executions>
<execution>
Expand Down
101 changes: 101 additions & 0 deletions guava/javadoc-link/checker-framework/package-list
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
android.annotation
android.support.annotation
com.sun.istack.internal
edu.umd.cs.findbugs.annotations
javax.annotation
javax.annotation.concurrent
javax.annotation.meta
javax.validation.constraints
lombok
net.jcip.annotations
org.checkerframework.checker.compilermsgs
org.checkerframework.checker.compilermsgs.qual
org.checkerframework.checker.fenum
org.checkerframework.checker.fenum.qual
org.checkerframework.checker.formatter
org.checkerframework.checker.formatter.qual
org.checkerframework.checker.guieffect
org.checkerframework.checker.guieffect.qual
org.checkerframework.checker.i18n
org.checkerframework.checker.i18n.qual
org.checkerframework.checker.i18nformatter
org.checkerframework.checker.i18nformatter.qual
org.checkerframework.checker.i18nformatter.unittests
org.checkerframework.checker.index
org.checkerframework.checker.index.lowerbound
org.checkerframework.checker.index.qual
org.checkerframework.checker.index.samelen
org.checkerframework.checker.index.searchindex
org.checkerframework.checker.index.substringindex
org.checkerframework.checker.index.upperbound
org.checkerframework.checker.initialization
org.checkerframework.checker.initialization.qual
org.checkerframework.checker.interning
org.checkerframework.checker.interning.qual
org.checkerframework.checker.linear
org.checkerframework.checker.linear.qual
org.checkerframework.checker.lock
org.checkerframework.checker.lock.qual
org.checkerframework.checker.nullness
org.checkerframework.checker.nullness.compatqual
org.checkerframework.checker.nullness.qual
org.checkerframework.checker.propkey
org.checkerframework.checker.propkey.qual
org.checkerframework.checker.regex
org.checkerframework.checker.regex.qual
org.checkerframework.checker.signature
org.checkerframework.checker.signature.qual
org.checkerframework.checker.signedness
org.checkerframework.checker.signedness.qual
org.checkerframework.checker.tainting
org.checkerframework.checker.tainting.qual
org.checkerframework.checker.units
org.checkerframework.checker.units.qual
org.checkerframework.common.aliasing
org.checkerframework.common.aliasing.qual
org.checkerframework.common.basetype
org.checkerframework.common.reflection
org.checkerframework.common.reflection.qual
org.checkerframework.common.subtyping
org.checkerframework.common.util
org.checkerframework.common.util.count
org.checkerframework.common.util.debug
org.checkerframework.common.util.report
org.checkerframework.common.util.report.qual
org.checkerframework.common.value
org.checkerframework.common.value.qual
org.checkerframework.common.value.util
org.checkerframework.common.wholeprograminference
org.checkerframework.dataflow.analysis
org.checkerframework.dataflow.cfg
org.checkerframework.dataflow.cfg.block
org.checkerframework.dataflow.cfg.node
org.checkerframework.dataflow.cfg.playground
org.checkerframework.dataflow.constantpropagation
org.checkerframework.dataflow.qual
org.checkerframework.dataflow.util
org.checkerframework.framework.flow
org.checkerframework.framework.qual
org.checkerframework.framework.source
org.checkerframework.framework.test
org.checkerframework.framework.test.diagnostics
org.checkerframework.framework.type
org.checkerframework.framework.type.treeannotator
org.checkerframework.framework.type.typeannotator
org.checkerframework.framework.type.visitor
org.checkerframework.framework.util
org.checkerframework.framework.util.defaults
org.checkerframework.framework.util.dependenttypes
org.checkerframework.framework.util.element
org.checkerframework.framework.util.typeinference
org.checkerframework.framework.util.typeinference.constraint
org.checkerframework.framework.util.typeinference.solver
org.checkerframework.javacutil
org.checkerframework.javacutil.dist
org.checkerframework.javacutil.trees
org.eclipse.jdt.annotation
org.eclipse.jgit.annotations
org.jetbrains.annotations
org.jmlspecs.annotation
org.netbeans.api.annotations.common
org.springframework.lang
25 changes: 14 additions & 11 deletions guava/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -108,16 +108,19 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<configuration>
<encoding>UTF-8</encoding>
<docencoding>UTF-8</docencoding>
<charset>UTF-8</charset>
<additionalparam>-XDignore.symbol.file</additionalparam>
<additionalparam>-Xdoclint:-html</additionalparam>
<!-- TODO(cpovirk): Move this to the parent after making jdk-sources available there. -->
<!-- TODO(cpovirk): can we use includeDependencySources and a local com.oracle.java:jdk-lib:noversion:sources instead of all this unzipping and manual sourcepath modification? -->
<!-- (We need JDK *sources*, not just -link, so that {@inheritDoc} works.) -->
<sourcepath>${project.build.sourceDirectory}:${project.build.directory}/jdk-sources</sourcepath>

<subpackages>com.google.common</subpackages>
<excludePackageNames>com.google.common.base.internal</excludePackageNames>
<linksource>true</linksource>

<!-- TODO(cpovirk): Move this to the parent after making the package-list files available there. -->
<!-- We add the link ourselves, both so that we can choose Java 9 over the version that -source suggests and so that we can solve the JSR305 problem described below. -->
<detectJavaApiLink>false</detectJavaApiLink>
<offlineLinks>
<!-- Even after we stop using JSR305 annotations in our own code, we'll want this link so that NullPointerTester's docs can link to @CheckForNull and friends... at least once we start using this config for guava-testlib. -->
<offlineLink>
<url>https://static.javadoc.io/com.google.code.findbugs/jsr305/3.0.1/</url>
<location>${project.basedir}/javadoc-link/jsr305</location>
Expand All @@ -131,15 +134,15 @@
<url>https://docs.oracle.com/javase/9/docs/api/</url>
<location>https://docs.oracle.com/javase/9/docs/api/</location>
</offlineLink>
<!-- The Checker Framework likewise would claim javax.annotations, despite providing only a subset of the JSR305 annotations, so it must likewise come after JSR305. -->
<offlineLink>
<url>https://checkerframework.org/api/</url>
<location>${project.basedir}/javadoc-link/checker-framework</location>
</offlineLink>
</offlineLinks>
<links>
<link>http://errorprone.info/api/latest/</link>
<link>https://checkerframework.org/api/</link>
</links>
<!-- TODO(cpovirk): can we use includeDependencySources and a local com.oracle.java:jdk-lib:noversion:sources instead of all this unzipping and manual sourcepath modification? -->
<!-- (We need JDK *sources*, not just -link, so that {@inheritDoc} works.) -->
<sourcepath>${project.build.sourceDirectory}:${project.build.directory}/jdk-sources</sourcepath>
<subpackages>com.google.common</subpackages>
</configuration>
<executions>
<execution>
Expand Down
6 changes: 6 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,12 @@
<configuration>
<notimestamp>true</notimestamp>
<additionalparam>-Xdoclint:-html</additionalparam>
<encoding>UTF-8</encoding>
<docencoding>UTF-8</docencoding>
<charset>UTF-8</charset>
<additionalparam>-XDignore.symbol.file</additionalparam>
<additionalparam>-Xdoclint:-html</additionalparam>
<linksource>true</linksource>
</configuration>
<executions>
<execution>
Expand Down

0 comments on commit 76f93b3

Please sign in to comment.