Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bubblewrap fails on sites that use HTTP/2 #516

Closed
JudahGabriel opened this issue May 3, 2021 · 5 comments
Closed

Bubblewrap fails on sites that use HTTP/2 #516

JudahGabriel opened this issue May 3, 2021 · 5 comments
Labels
depends-external Depends on changes on an external project, android-browser-helper or the underlying browsers

Comments

@JudahGabriel
Copy link
Contributor

JudahGabriel commented May 3, 2021 •
edited
Loading

When generating a TWA for a site that uses HTTP/2, Bubblewrap fails with the following error:

021-05-03T18:34:11.907964631Z: [ERROR] Responded with status 500
2021-05-03T18:34:11.907985832Z: [ERROR] at TwaGenerator.writeWebManifest (/app/node_modules/@bubblewrap/core/dist/lib/TwaGenerator.js:218:19)
2021-05-03T18:34:11.907993432Z: [ERROR] at runMicrotasks ()
2021-05-03T18:34:11.907998132Z: [ERROR] at processTicksAndRejections (internal/process/task_queues.js:97:5)
2021-05-03T18:34:11.908002332Z: [ERROR] at TwaGenerator.createTwaProject (/app/node_modules/@bubblewrap/core/dist/lib/TwaGenerator.js:344:13)

Steps to reproduce the behavior:
Run Bubblewrap on a site that serves HTTP/2 content, such as https://jued.ru. This can be done from PWABuilder as well. You'll receive the above error.

I think the underlying issue is that Bubblewrap uses node-fetch, and node-fetch doesn't support HTTP/2. We've had several bugs opened on PWABuilder where the user gets the above error when creating a TWA package. Upon investigation, the issue is the site is using HTTP/2. Original bug report here.

Most of the times we see this error, it's because the developer enabled Cloudflare, which allows HTTP/2 (and HTTP/3!) with the flip of a switch.
@JudahGabriel JudahGabriel mentioned this issue May 3, 2021
Closed
@andreban
Copy link
Member

andreban commented May 4, 2021

Hey @JudahGabriel

In the stacktrace above, we can see 021-05-03T18:34:11.907964631Z: [ERROR] Responded with status 500 - this indicates the server returned an Internal Server Error, not an HTTP/2 error. Unfortunately, it seems the error is on the developers server.

Here's the output of curl -v https://jued.ru/manifest.json, which returns a 403 error:

Expand to see the output: 
❯ curl -v https://jued.ru/manifest.json
*   Trying 104.26.11.228...
* TCP_NODELAY set
* Connected to jued.ru (104.26.11.228) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-CHACHA20-POLY1305
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.jued.ru
*  start date: Mar  6 02:29:37 2021 GMT
*  expire date: Jun  4 02:29:37 2021 GMT
*  subjectAltName: host "jued.ru" matched cert's "jued.ru"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fe31c00d600)
> GET /manifest.json HTTP/2
> Host: jued.ru
> User-Agent: curl/7.64.1
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 403 
< date: Tue, 04 May 2021 08:47:20 GMT
< content-type: text/html; charset=UTF-8
< cf-chl-bypass: 1
< set-cookie: __cfduid=df8842da35e17c9b31266a7ac372201501620118040; expires=Thu, 03-Jun-21 08:47:20 GMT; path=/; domain=.jued.ru; HttpOnly; SameSite=Lax; Secure
< cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< expires: Thu, 01 Jan 1970 00:00:01 GMT
< x-frame-options: SAMEORIGIN
< cf-request-id: 09d828865300000081b983a000000001
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=I8bq9Xhd0jW3XMs4ywJtK3tw9qxhVHWhcmL9CLZZb21BpRqubKorTdW9wjFILsovLQBx16%2FXxwKZYOcHxbdrNbnW0a407GG8"}],"max_age":604800,"group":"cf-nel"}
< nel: {"max_age":604800,"report_to":"cf-nel"}
< x-content-type-options: nosniff
< server: cloudflare
< cf-ray: 64a076b6eee20081-LHR
< alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400
< 
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>

<title>Please Wait... | Cloudflare</title>
  
<meta name="captcha-bypass" id="captcha-bypass" />
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/cf.errors.css" type="text/css" media="screen,projection" />
<!--[if lt IE 9]><link rel="stylesheet" id='cf_styles-ie-css' href="/cdn-cgi/styles/cf.errors.ie.css" type="text/css" media="screen,projection" /><![endif]-->
<style type="text/css">body{margin:0;padding:0}</style>


<!--[if gte IE 10]><!-->
<script>
  if (!navigator.cookieEnabled) {
    window.addEventListener('DOMContentLoaded', function () {
      var cookieEl = document.getElementById('cookie-alert');
      cookieEl.style.display = 'block';
    })
  }
</script>
<!--<![endif]-->


  
    <script type="text/javascript">
    //<![CDATA[
    (function(){
      window._cf_chl_opt={
        cvId: "2",
        cType: "managed",
        cNounce: "80824",
        cRay: "64a076b6eee20081",
        cHash: "6e64cd88ca71710",
        cFPWv: "b",
        cTTimeMs: "4000",
        cLt: "n",
        cRq: {
          ru: "aHR0cHM6Ly9qdWVkLnJ1L21hbmlmZXN0Lmpzb24=",
          ra: "Y3VybC83LjY0LjE=",
          rm: "R0VU",
          d: "QqN/J7ORPI78SOcLxUAlZSf7VRfHku4RyywJpGSG7efbr9y4QUoS11f/qXVmnm3EZxnkdRTEKUE+8hpuwLka64hOKM3y0Tgt2MebEEZyXFo3mpgjqLyhxC18ra6HJ52jUEigdgDK75xRx4fExfKB+7WBHeY9AQDq5bY42xkYVK7pZqOzFmZQO9+65vL6cXnkMK/WX/8mDKJFf9ZhCvDw1Isj3TrWqyXesv4rZv50PJVnKe1spAJqiMWVSgtOFFVt6XCP2ZeXJbch+ba0jiyra6L8KmyX7eqgQFvgMJq7m3Q4npNUd6nw3p2pKsgwLrXWlG5vD3w1uSnANJ1M7oArfmnB26x1VXidmPythHkRN3s1H1xJOQX5EyddNHRwd7951pnd8BAuqOf4/1VZihl3R0m9ljCAr4K+iwjJice/jWrxzckT5yhY0Dnvv2KRmqs+jcLyvHsbJet1kMTnv/RL+kletioTxL+QezNzSWra0ZcrpP74w54h8qGEgeXqY3DVSLQyukZnKTRCwkyrUK942p+Al8YGf7/WfglMjrJKsRxyJD2WWOKT5ePZpQKRuG2L4NHzIf1UiI/Or66PvQ+NptpHdKC/YapTW8sIwudDXCcNnqU0WTRgCpS2tI2oq+Sqc4mjpxzVl/3JbuMgq579UYZe+w778yygN21fki2aVjvFS7U2uBvALND8DxU9xlKr4ZO28QQ/bO7Nv5VA2JqqAM7FXfhKokGEcyGJiyLP5ECPS4+EaQBZ2FJoUExZGO51OE1n94FTzEjNR/o9qHlqZa+Yy+Iny3UHS8s4GSbWJf7XE+174Wfyoi5xdIGNKWJzDyX4PruSV9a7U1rsvkfo2vT8iQA4LisULfdd9atVRNe/kAQm5122Euf7X0oG6V8dwgKOfj0ENzEaa5Po2lkZH+4IQBXjb5YJ1glZWjKDrT8=",
          t: "MTYyMDExODA0MC4xNTkwMDA=",
          m: "BZ4CVMDcFw8bLJ6muHygMxeaP6mnpgD98dxo9Voo56c=",
          i1: "iQwD7SSra1B9C+pEbZ95OA==",
          i2: "sp/rZRFHJ76fRv3aovyOEA==",
          zh: "tBmyPuiDhSKV9sLpFEHW8xnhOpV4lrNh5MQlP6FvOig=",
          uh: "szTQexRnAk6IzW67QzqGQgGxkzC/vAX/8LMiqwBnOyk=",
          hh: "tLK1mOEDTr8MbVeLF4FXrXdLYUNFkCBejeaS/zmuFYk=",
        }
      };
    }());
    //]]>
    </script>
  

<style type="text/css">
  #cf-wrapper #spinner {width:69px; margin:  auto;}
  #cf-wrapper #cf-please-wait{text-align:center}
  .attribution {margin-top: 32px;}
  .bubbles { background-color: #f58220; width:20px; height: 20px; margin:2px; border-radius:100%; display:inline-block; }
  #cf-wrapper #challenge-form { padding-top:25px; padding-bottom:25px; }
  #cf-hcaptcha-container { text-align:center;}
  #cf-hcaptcha-container iframe { display: inline-block;}
  @keyframes fader     { 0% {opacity: 0.2;} 50% {opacity: 1.0;} 100% {opacity: 0.2;} }
  #cf-wrapper #cf-bubbles { width:69px; }
  @-webkit-keyframes fader { 0% {opacity: 0.2;} 50% {opacity: 1.0;} 100% {opacity: 0.2;} }
  #cf-bubbles > .bubbles { animation: fader 1.6s infinite;}
  #cf-bubbles > .bubbles:nth-child(2) { animation-delay: .2s;}
  #cf-bubbles > .bubbles:nth-child(3) { animation-delay: .4s;}
</style>
</head>
<body>
  <div id="cf-wrapper">
    <div class="cf-alert cf-alert-error cf-cookie-error" id="cookie-alert" data-translate="enable_cookies">Please enable cookies.</div>
    <div id="cf-error-details" class="cf-error-details-wrapper">
      <div class="cf-wrapper cf-header cf-error-overview">
      
        <h1 data-translate="managed_challenge_headline">Please wait...</h1>
        <h2 class="cf-subheadline"><span data-translate="managed_checking_msg">We are checking your browser...</span> jued.ru</h2>
      
      </div>
      
      <div class="cf-section cf-highlight cf-captcha-container">
        <div class="cf-wrapper">
          <div class="cf-columns two">
            <div class="cf-column">
            
              <div class="cf-highlight-inverse cf-form-stacked">
                <form class="challenge-form managed-form" id="challenge-form" action="/manifest.json?__cf_chl_managed_tk__=9e7993cf066743e981fddfdf0d8936a8b584cfd5-1620118040-0-AY3Mvlt1nCRjeEEieVd2ezVCbqvKch5XAaqrsdZ9UzF5o6V9nkGv3b9sc6u9FLP6lvSUrbncUgG1eSDftVYAPISf3WvEfL33nsd4_Fw8_euneV31vfOtbrPLeSxPUh1FAegxzfUyVxG4e0bN7QKW63SvmwSEc2Ig5UcbKovmasOLw6Uj5TXlDMbasT-qa0GzmSmtzEJwW7DMgLJMKZ0ERh8KxnF27Z8hiWsBGHOu6yDhJ8zkSf4p0ykEmVhRPUI0i5DnpxtiY-BL5K3qhUCdj1JmyTIJhzce7ylq-FkPbmVPvrbL1mr3ZiDM9ObS94DiKZp7eeHKMUaln5OBwGyMyG9E4TB6oC416Xt7p8q9zbVCWrCqAtC2_EwTOhicrdXgFsM2QUPE6IY4PuuJauajtRUM18uRNQ5qccRdQmpaMjD4luBtD4CZRqkjMS7qOclqbck_QTS2eZrgcR-y7z5vEUmffDCXzvcQtUzk_qp93w4DM_heAmeiyV9jFMVs6WPs1rLq4svjB2rBeuNnzLwUQ1mh8V5cXcgr-7JVbLKDpzM_69Q5IAUFAD3mCws3whsfMYu3wH4-4L1P0L86ijxgy0fHofTTvCDEnXWtJwD-_zYo8B0AZF5LgszPEnrEqRi8z2NE3Cd8byiVZSp2IuSEuJH5d9F_Xq7Erk09C53tTaZC4JgBywnn5NcX_NGiqZYqaA" method="POST" enctype="application/x-www-form-urlencoded">
  
    <div id='cf-please-wait'>
      <div id='spinner'>
        <div id="cf-bubbles">
            <div class="bubbles"></div>
            <div class="bubbles"></div>
            <div class="bubbles"></div>
        </div>
      </div>
      <p data-translate="please_wait" id="cf-spinner-please-wait">Please stand by, while we are checking your browser...</p>
      <p data-translate="redirecting" id="cf-spinner-redirecting" style="display:none">Redirecting...</p>
      </div>
  
  <input type="hidden" name="r" value="22426e6864b6f2064763a36d3178aff61dd2b178-1620118040-0-AROZ0eCARrDp2eCxOnI1lNoPvBtogE8+3219Zafruhquyw+vQCgHrGmF98uLLJH4tRALnC0IlD+epa/uNt/UttoVG+3fjM0ufIC97bxB8ndaPZRO/IEoJJsFRaq7oXg5w7+8Y2xhcO+VQR5ypNmwonsV4VF0Q2bf/nI17bjP1BgkF1culZ/T9buK9h+RbFpjmdTt+JYvw7SVs5rNgMOV6FyHX7amcfvFFwlNUhio+zZFZI8QJqJgQRsoWiAI9z2fpHp7L1xyM1eYA3ejmTkPs1Za4DPJIbc7isIEcveLLGcL9lpMSVbeddNOkRAL1E3L7ls8P7OZ7x1rpZP8q4l4PqWvjYgpxyJNXa8SOXaKaXdaYv4HTwcDIvvoJGXfjN/2P3CZft+cxaTKzmG38WqIMB6SAmTp4r/tXPF6DsM2mQs5/JymodraWDrYzyDdcIBGbXerSw7OzXMs8In3+VAfQ7xk1ppd5aKsbl1bQZP1ZPBz+4WCAsZ0uYn/Uayp0hynN50reTTf+pQ+7TdATEmsibS9JUNsLQrvMSuLko3zvm+KEZsw1EqE3iv+3/yeNlb9+bYNpvJgf2xodKkHewk0EjLqwomhdMM7tV+ZFQzCCnHATeVe+SJRXzFMm5rm1emPISMcGe9cga8l+zkPXPddyXpFr7jXWjNc4wd/Uyo48j/SeAO06LClQXsYWWw7Vj50KJGcM/gRiOelKeKrOJzp1t2X7cypSUUQP9DKeuYXx4QDcvGpd3s1FPC3nhsO2THOUr1MuFZpyl7Z4cfDriFEKVLoTtusDehPQJtXfQi6UTxVVrTdZGhDJOBuqctxCW5ymQCjoLYtswg1M7LWua1q9dnPWs6R+ZAAn721IOX4fHrM5qyiVANu9ZDgQvEGu6ipTaG93J/jl86Q3oHAqSujd0Av2hYdBUh7I1kN1F8IlIFp6pV+FGxRKu72M+g2e8lGAjCo48GkOOReZBkFZMhlbuTwNcwOlEwiTsz7c7eOtT6ZsweBdMwGYcPuUYQxKeNywKUkw/UFOehONOOwpMFzsMBDZ7KHCvWvmYELCPAZeTTXvGdnuuroipZ/AYmVPMii6bMXjbhYUCkDh/9VKOv/Dw7RRRjxPF1Q0cFLaZbe0P2Berov9mp++j4pKpeNSmN1DNTJPEC207tbCPg5Urln3B8LUR6ja1dRUZl8vcCu+O5nsVZYP8mJCVSUAqU6obHq4aV00c616BWqLPyaZZd5Mg0n4VtY1zz4CZDSWV8pV0rAQSm26xDMQnTyrRs000AmKWy1dXMNeKLaYu7AmA0KO3g=">
  <input type="hidden" name="cf_captcha_kind" value="h">
  <input type="hidden" name="vc" value="a76972929c3911b4924dc5a6fee54173">
  
  <noscript id="cf-captcha-bookmark" class="cf-captcha-info">
  <h1 data-translate="turn_on_js" style="color:#bd2426;">Please turn JavaScript on and reload the page.</h1>
  </noscript>
    <div id="no-cookie-warning" class="cookie-warning" data-translate="turn_on_cookies" style="display:none">
      <p data-translate="turn_on_cookies" style="color:#bd2426;">Please enable Cookies and reload the page.</p>
    </div>
  <script type="text/javascript">
  //<![CDATA[
    var a = function() {try{return !!window.addEventListener} catch(e) {return !1} },
      b = function(b, c) {a() ? document.addEventListener("DOMContentLoaded", b, c) : document.attachEvent("onreadystatechange", b)};
      b(function(){
        var cookiesEnabled=(navigator.cookieEnabled)? true : false;
        if(!cookiesEnabled){
          var q = document.getElementById('no-cookie-warning');q.style.display = 'block';
        }
      });
  //]]>
  </script>
  <div id="trk_captcha_js" style="background-image:url('/cdn-cgi/images/trace/captcha/nojs/h/transparent.gif?ray=64a076b6eee20081')"></div>
</form>
  
  <script type="text/javascript">
    //<![CDATA[
    (function(){
        var isIE = /(MSIE|Trident\/|Edge\/)/i.test(window.navigator.userAgent);
        var trkjs = isIE ? new Image() : document.createElement('img');
        trkjs.setAttribute("src", "/cdn-cgi/images/trace/managed/js/transparent.gif?ray=64a076b6eee20081");
        trkjs.id = "trk_managed_js";
        trkjs.setAttribute("alt", "");
        document.body.appendChild(trkjs);
        var cpo=document.createElement('script');
        cpo.type='text/javascript';
        cpo.src="/cdn-cgi/challenge-platform/h/b/orchestrate/managed/v1?ray=64a076b6eee20081";
        document.getElementsByTagName('head')[0].appendChild(cpo);
    }());
    //]]>
    </script>
  


              </div>
            </div>

            <div class="cf-column">
              <div class="cf-screenshot-container">
              
                <span class="cf-no-screenshot"></span>
              
              </div>
            </div>
          </div>
        </div>
      </div>

      <div class="cf-section cf-wrapper">
        <div class="cf-columns two">
          <div class="cf-column">
            <h2 data-translate="why_captcha_headline">Why do I have to complete a CAPTCHA?</h2>
            
            <p data-translate="why_captcha_detail">Completing the CAPTCHA proves you are a human and gives you temporary access to the web property.</p>
          </div>

          <div class="cf-column">
            <h2 data-translate="resolve_captcha_headline">What can I do to prevent this in the future?</h2>
            

            <p data-translate="resolve_captcha_antivirus">If you are on a personal connection, like at home, you can run an anti-virus scan on your device to make sure it is not infected with malware.</p>

            <p data-translate="resolve_captcha_network">If you are at an office or shared network, you can ask the network administrator to run a scan across the network looking for misconfigured or infected devices.</p>
            
              
            
          </div>
        </div>
      </div>
      

      <div class="cf-error-footer cf-wrapper w-240 lg:w-full py-10 sm:py-4 sm:px-8 mx-auto text-center sm:text-left border-solid border-0 border-t border-gray-300">
  <p class="text-13">
    <span class="cf-footer-item sm:block sm:mb-1">Cloudflare Ray ID: <strong class="font-semibold">64a076b6eee20081</strong></span>
    <span class="cf-footer-separator sm:hidden">&bull;</span>
    <span class="cf-footer-item sm:block sm:mb-1"><span>Your IP</span>: 77.96.185.83</span>
    <span class="cf-footer-separator sm:hidden">&bull;</span>
    <span class="cf-footer-item sm:block sm:mb-1"><span>Performance &amp; security by</span> <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing" id="brand_link" target="_blank">Cloudflare</a></span>
    
  </p>
</div><!-- /.error-footer -->


    </div>
  </div>

  <script type="text/javascript">
  window._cf_translation = {};
  
  
</script>


</body>
</html>
* Connection #0 to host jued.ru left intact
* Closing connection 0

@andreban
Copy link
Member

andreban commented May 4, 2021

Adding a bit more context here. node-fetch, they underlying library for requests, doesn't support HTTP/2 (see node-fetch/node-fetch#342).

Generally, this shouldn't be a big issue since, as a best practice, servers should be configured to do HTTP/2 Upgrade instead of HTTP/2 only, as many clients don't support it yet.

I see the server in questions is currently configured to do upgrades:

* ALPN, offering h2
* ALPN, offering http/1.1

@JudahGabriel
Copy link
Contributor Author

JudahGabriel commented May 4, 2021 •
edited
Loading

Interesting. So this is a combination of using HTTP/2 (node-fetch doesn't yet support it) and the web server blocking access?

If the web server is blocking access, why can the browser access it from the same IP?

@andreban
Copy link
Member

andreban commented May 4, 2021

From my curl test, it seems the server supports HTTP/1.1, but is blocking access (over H2 too, as curl uses H2).

@andreban andreban added the depends-external Depends on changes on an external project, android-browser-helper or the underlying browsers label May 6, 2021
@andreban andreban mentioned this issue May 7, 2021
Merged
@andreban
Copy link
Member

andreban commented Jun 3, 2021

Closed via #522 and published in v1.13.2.

@andreban andreban closed this as completed Jun 3, 2021
@nohe427 nohe427 mentioned this issue Jan 13, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
depends-external Depends on changes on an external project, android-browser-helper or the underlying browsers
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JudahGabriel @andreban