-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update protobuf/grpc/tcnative dependencies #1691
Comments
Also do googleapis/gax-java#186 while we're at it (move from grpc-all to more minimal grpc dependency set) |
I'd suggest updating netty-tcnative to 2.x as per netty/netty#5766 Right now it is blocking using 0.9.4-alpha (or 0.8.0 or 0.7.0) in Linux+Tomcat / SpringBoot+TomcatEmbeddedServletContainer / or Elastic Beanstalk + Tomcat8 |
@kinow , Is it blocking because of conflicting netty versions? What version of netty does your version of Tomcat depend on? |
I have successfully deployed the application using Jetty, and packaging it as an executable war. I'm in the process of creating a smaller example that can be uploaded to GitHub, and also a blog post about it. My guess is that google-cloud-java is using a version of netty lower than 2.x. Because of that this application is loading a version of netty-tcnative-boringssl-static that loads a class in a package from Apache Tomcat. What happens in this application, is that it successfully starts up with Spring Boot, then during my first request it will create the authentication tokens and submit the first request. This request is simply never processed, and after a few moments I will see a DEADLINE_EXCEEDED exception (IIRC, at home right now). I believe that updating to netty-tcnative 2.x, this issue will be gone, and google-cloud-java should be fully compatible with Spring Boot + Tomcat. |
Here's an example project. https://github.com/kinow/gs-spring-boot/tree/reproduce-google-nlp-issue/complete Here's the diff: spring-guides/gs-spring-boot@master...kinow:reproduce-google-nlp-issue I forked the Spring Boot Getting Started sample application, which is using the latest version of Spring Boot. Created a branch, and added google-cloud-language 0.9.4-alpha (used the sample code from https://cloud.google.com/natural-language/docs/samples). If you start the original project with
In the branch pom.xml, I added the entries to use Jetty instead of Tomcat embedded container. Uncommenting those lines and running again with Adding an exclusion to io.netty:netty-tcnative-boringssl-static to google-cloud-language does not work. The issue persists, and you still get a DEADLINE EXCEEDED exception. I've cloned the google-cloud-java locally. Updated all dependencies as suggested in this issue, except for Grpc which was breaking the tests, installed a snapshot locally, and updated the example to use google-cloud-java:0.9.5-alpha-SNAPSHOT. All tests in google-cloud-java passed, and the application also started up successfully, but the issue persisted. Simply updating netty-tcnative to 2.x won't fix the issue. My guess now is that with a version of grpc-netty (and hence a newer version of netty, compatible with netty-tcnative 2.x) the issue may be fixed. Will file a new issue for that, once I have some more time to debug the problem. Sorry for hijacking this issue. Bruno |
This is blocked on artman updates to fix grpc package generation (googleapis/gapic-generator#803). cc @shinfan |
@kinow
So while an updated tcnative might fix your tomcat issues, it won't fix any |
You could be right. If I find time next week I will debug what happens in the Google Cloud NLP API. So far what I noticed is that when using Tomcat with Google Cloud NLP, I can see that it
When I replace Tomcat by Jetty, the three first steps are successfully executed. And then the fourth step (where The first time I debugged in Eclipse I noticed either in the console or logs, a message with a Google Cloud URL and the 443 (can't really remember off the top of my head, but it was something like "opening connection to cloud.google.com 443"). Next time I debug it, I will work with the hypothesis that when I use Tomcat; some thing (interface/class/service loader/native library/etc) provided by both netty-tcnative and Tomcat fails when it tries to open a SSL connection to the google cloud server. Then either the connection is never successfully established, or maybe something else happens, and the Future class times out. First thing I will do is probably fire Wireshark and just see what client / server are sending with Jetty and with Tomcat. Cheers |
Had some time before going to sleep. Running the sample application with Tomcat embedded container, I still get the So client is sending the packages to the server. For some reason it is getting stuck and timing out. |
cc @ejona86 here is another example of the DEADLINE_EXCEEDED issue. @buchgr and @kinow , this is very likely caused by an issue with loading netty-tcnative, similar to #1430 and #1772 . Basically when netty-tcnative doesn't load right, there is some misbehavior somewhere and grpc accepts the request but it never successfully gets issued, so the only thing that stops it is whatever timeout is set. I filed issue grpc/grpc-java#2599 against grpc-java asking for them to make the error messaging more accurate. To be sure, you need to set your logging level to FINE to see if you see logs that say |
Thanks @garrettjonesgoogle Note to self, check the logs :-) Here's the output after adding a logback file like this:
Omitting only the access settings
|
@shinfan when you do this in the generated packages, at the same time it would be nice to convert from depending on grpc-all to depending on more granular grpc packages. |
@garrettjonesgoogle Updated. Do we want to close this now? Or are there pending items? |
When there is a new release of google-cloud-java, I can try building the language-api and test it (or even better if there was an updated release of language-api). Feel free to close this issue in the meantime, will comment here with the result of the testing for posteriority |
It looks like the tcnative dependency didn't get updated - it seems that this time, the newer grpc still works with the previous tcnative. In any case, I think it would be prudent to update it before releasing. |
I have updated the tcnative dependency. Closing out this issue now. |
thanks @garrettjonesgoogle and @pongad |
…essor job (#1691) (#123) * chore: install dependencies through requirements file Source-Link: https://togithub.com/googleapis/synthtool/commit/35f4cbaf1295a726cb43fd4471129ec74b48e04e Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:821ab7aba89af2c7907e29297bba024d4cd5366d0684e5eb463391cdf4edc9ee
…essor job (#1691) (#768) * chore: install dependencies through requirements file Source-Link: https://togithub.com/googleapis/synthtool/commit/35f4cbaf1295a726cb43fd4471129ec74b48e04e Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:821ab7aba89af2c7907e29297bba024d4cd5366d0684e5eb463391cdf4edc9ee
…essor job (#1691) (#1007) * chore: install dependencies through requirements file Source-Link: https://togithub.com/googleapis/synthtool/commit/35f4cbaf1295a726cb43fd4471129ec74b48e04e Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:821ab7aba89af2c7907e29297bba024d4cd5366d0684e5eb463391cdf4edc9ee
Grpc: Now = 1.0.3; update to 1.2.0
Protobuf: Now = 3.0.0 & 3.0.2 (depending on where); update to 3.2.0
netty-tcnative: Now = 1.1.33.Fork19; update to 1.1.33.Fork26
Note: this also involves updating the generated grpc packages and gax.
The text was updated successfully, but these errors were encountered: