install-moonraker.sh failing due to html parser error

[semicuda]

Output below. Adding "--use-deprecated=html5lib" to install-moonraker.sh produces this output:

###### Removing old virtualenv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/pi/moonraker-env/bin/python3
Also creating executable in /home/pi/moonraker-env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
ERROR: Could not find a version that satisfies the requirement tornado==6.1.0 (from versions: none)
ERROR: No matching distribution found for tornado==6.1.0

Original output:

pi@fluiddpi:~/moonraker $ ./scripts/install-moonraker.sh -r -f


###### Running apt-get update...
Hit:1 http://raspbian.raspberrypi.org/raspbian buster InRelease
Hit:2 http://archive.raspberrypi.org/debian buster InRelease
Reading package lists... Done


###### Installing packages...
Reading package lists... Done
Building dependency tree
Reading state information... Done
curl is already the newest version (7.64.0-4+deb10u2).
libcurl4-openssl-dev is already the newest version (7.64.0-4+deb10u2).
libjpeg-dev is already the newest version (1:1.5.2-2+deb10u1).
liblmdb-dev is already the newest version (0.9.22-1).
libopenjp2-7 is already the newest version (2.3.0-2+deb10u2).
libsodium-dev is already the newest version (1.0.17-1).
libssl-dev is already the newest version (1.1.1d-0+deb10u7).
python3-dev is already the newest version (3.7.3-1).
python3-libgpiod is already the newest version (1.2-3+rpi1).
python3-virtualenv is already the newest version (15.1.0+ds-2).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-1).
The following package was automatically installed and is no longer required:
  python-colorzero
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.


###### Installing python virtual environment...


###### Removing old virtualenv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/pi/moonraker-env/bin/python3
Also creating executable in /home/pi/moonraker-env/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
ERROR: Exception:
Traceback (most recent call last):
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 165, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 340, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 44, in _iter_built
    for version, func in infos:
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 297, in iter_index_candidate_infos
    hashes=hashes,
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 868, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 809, in find_all_candidates
    page_candidates = list(page_candidates_it)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/sources.py", line 134, in page_candidates
    yield from self._candidates_from_page(self._link)
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 773, in process_project_url
    page_links = list(parse_links(html_page, self._use_deprecated_html5lib))
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 310, in wrapper_wrapper
    return list(fn(page, use_deprecated_html5lib))
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 350, in parse_links
    parser.feed(page.content.decode(encoding))
  File "/usr/lib/python3.7/html/parser.py", line 111, in feed
    self.goahead(0)
  File "/usr/lib/python3.7/html/parser.py", line 179, in goahead
    k = self.parse_html_declaration(i)
  File "/usr/lib/python3.7/html/parser.py", line 270, in parse_html_declaration
    self.handle_decl(rawdata[i+2:gtpos])
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 405, in handle_decl
    self._raise_error()
  File "/home/pi/moonraker-env/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 427, in _raise_error
    "HTML doctype missing or incorrect. Expected <!DOCTYPE html>.\n\n"
ValueError: HTML doctype missing or incorrect. Expected <!DOCTYPE html>.

If you believe this error to be incorrect, try passing the command line option --use-deprecated=html5lib and please leave a comment on the pip issue at https://github.com/pypa/pip/issues/10825.

[Arksine]

This appears to be an upstream issue, see pyca/10825. Hopefully they will have it resolved soon.

[cbc02009]

fwiw, I can't reproduce this issue with python 3.9 and pip 22.0. Not sure what's different about mine that makes it work, but maybe it will help isolate where the issue is.

~/moonraker-env/bin/pip install --force-reinstall --no-cache-dir -r ~/moonraker/scripts/moonraker-requirements.txt
Collecting tornado==6.1.0
  Downloading tornado-6.1-cp39-cp39-manylinux2014_aarch64.whl (427 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 427.7/427.7 KB 1.5 MB/s eta 0:00:00
Collecting pyserial-asyncio==0.6
  Downloading pyserial_asyncio-0.6-py3-none-any.whl (7.6 kB)
Collecting pillow==9.0.0
  Downloading Pillow-9.0.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (4.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 11.1 MB/s eta 0:00:00
Collecting lmdb==1.2.1
  Downloading lmdb-1.2.1.tar.gz (881 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 881.5/881.5 KB 16.1 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting streaming-form-data==1.8.1
  Downloading streaming-form-data-1.8.1.tar.gz (90 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.5/90.5 KB 19.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting distro==1.5.0
  Downloading distro-1.5.0-py2.py3-none-any.whl (18 kB)
Collecting inotify-simple==1.3.5
  Downloading inotify_simple-1.3.5.tar.gz (9.7 kB)
  Preparing metadata (setup.py) ... done
Collecting libnacl==1.7.2
  Downloading libnacl-1.7.2-py2.py3-none-any.whl (20 kB)
Collecting paho-mqtt==1.5.1
  Downloading paho-mqtt-1.5.1.tar.gz (101 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.8/101.8 KB 19.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pycurl==7.44.1
  Downloading pycurl-7.44.1.tar.gz (227 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 227.6/227.6 KB 19.5 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting zeroconf==0.37.0
  Downloading zeroconf-0.37.0-py3-none-any.whl (105 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.9/105.9 KB 18.4 MB/s eta 0:00:00
Collecting preprocess-cancellation==0.1.6
  Downloading preprocess_cancellation-0.1.6-py3-none-any.whl (18 kB)
Collecting jinja2==3.0.3
  Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.6/133.6 KB 17.7 MB/s eta 0:00:00
Collecting dbus-next==0.2.3
  Downloading dbus_next-0.2.3-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.9/57.9 KB 16.3 MB/s eta 0:00:00
Collecting pyserial
  Downloading pyserial-3.5-py2.py3-none-any.whl (90 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.6/90.6 KB 16.1 MB/s eta 0:00:00
Collecting ifaddr>=0.1.7
  Downloading ifaddr-0.1.7-py2.py3-none-any.whl (10 kB)
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB)
Building wheels for collected packages: lmdb, streaming-form-data, inotify-simple, paho-mqtt, pycurl
  Building wheel for lmdb (setup.py) ... done
  Created wheel for lmdb: filename=lmdb-1.2.1-cp39-cp39-linux_aarch64.whl size=265996 sha256=65d25b808923db78a7d6cf1f15ebfc9e2313b17b91e0bf75957a20f78df23539
  Stored in directory: /tmp/pip-ephem-wheel-cache-q6f8d5x7/wheels/66/8c/02/44a2a6812b52f385b439e09a78776579dbd96e7a79ac297b5e
  Building wheel for streaming-form-data (pyproject.toml) ... done
  Created wheel for streaming-form-data: filename=streaming_form_data-1.8.1-cp39-cp39-linux_aarch64.whl size=322120 sha256=50f918fd8acbea560c914c455ba6bc473d9a18d5637d102f032b945f36f771de
  Stored in directory: /tmp/pip-ephem-wheel-cache-q6f8d5x7/wheels/6f/74/45/1a00df544d790c2dc2e919f9bec0707d7c2045d4e7b5a4bc04
  Building wheel for inotify-simple (setup.py) ... done
  Created wheel for inotify-simple: filename=inotify_simple-1.3.5-py3-none-any.whl size=7688 sha256=ad12057fb2e16402fca6a25fa7926c3b89ec6eef80d1fcd43d04b2e992de2aba
  Stored in directory: /tmp/pip-ephem-wheel-cache-q6f8d5x7/wheels/fa/bb/6f/f28b1936db58d9a92a165af1c633154c6264de022b3412de44
  Building wheel for paho-mqtt (setup.py) ... done
  Created wheel for paho-mqtt: filename=paho_mqtt-1.5.1-py3-none-any.whl size=61543 sha256=1aada792b0862deea6d513ca496bbdbb55274278f587c09ca5bb36482ca7a4b0
  Stored in directory: /tmp/pip-ephem-wheel-cache-q6f8d5x7/wheels/22/b9/0f/9a1f64674f849b8ae88620232f2023f0ff2a50a4479b8a32ed
  Building wheel for pycurl (setup.py) ... done
  Created wheel for pycurl: filename=pycurl-7.44.1-cp39-cp39-linux_aarch64.whl size=294230 sha256=b67cbb8dbff57d0e7e45e8ff8574ee88a710e39c6243725a5da45e26be2f9202
  Stored in directory: /tmp/pip-ephem-wheel-cache-q6f8d5x7/wheels/3e/c5/d9/41b90bcf9b14d27d2b74c2ae3c4789fe1ffdf9d17551ed143d
Successfully built lmdb streaming-form-data inotify-simple paho-mqtt pycurl
Installing collected packages: pyserial, paho-mqtt, lmdb, libnacl, ifaddr, distro, zeroconf, tornado, streaming-form-data, pyserial-asyncio, pycurl, preprocess-cancellation, pillow, MarkupSafe, inotify-simple, dbus-next, jinja2
  Attempting uninstall: pyserial
    Found existing installation: pyserial 3.5
    Uninstalling pyserial-3.5:
      Successfully uninstalled pyserial-3.5
  Attempting uninstall: paho-mqtt
    Found existing installation: paho-mqtt 1.5.1
    Uninstalling paho-mqtt-1.5.1:
      Successfully uninstalled paho-mqtt-1.5.1
  Attempting uninstall: lmdb
    Found existing installation: lmdb 1.2.1
    Uninstalling lmdb-1.2.1:
      Successfully uninstalled lmdb-1.2.1
  Attempting uninstall: libnacl
    Found existing installation: libnacl 1.7.2
    Uninstalling libnacl-1.7.2:
      Successfully uninstalled libnacl-1.7.2
  Attempting uninstall: ifaddr
    Found existing installation: ifaddr 0.1.7
    Uninstalling ifaddr-0.1.7:
      Successfully uninstalled ifaddr-0.1.7
  Attempting uninstall: distro
    Found existing installation: distro 1.5.0
    Uninstalling distro-1.5.0:
      Successfully uninstalled distro-1.5.0
  Attempting uninstall: zeroconf
    Found existing installation: zeroconf 0.37.0
    Uninstalling zeroconf-0.37.0:
      Successfully uninstalled zeroconf-0.37.0
  Attempting uninstall: tornado
    Found existing installation: tornado 6.1
    Uninstalling tornado-6.1:
      Successfully uninstalled tornado-6.1
  Attempting uninstall: streaming-form-data
    Found existing installation: streaming-form-data 1.8.1
    Uninstalling streaming-form-data-1.8.1:
      Successfully uninstalled streaming-form-data-1.8.1
  Attempting uninstall: pyserial-asyncio
    Found existing installation: pyserial-asyncio 0.6
    Uninstalling pyserial-asyncio-0.6:
      Successfully uninstalled pyserial-asyncio-0.6
  Attempting uninstall: pycurl
    Found existing installation: pycurl 7.44.1
    Uninstalling pycurl-7.44.1:
      Successfully uninstalled pycurl-7.44.1
  Attempting uninstall: preprocess-cancellation
    Found existing installation: preprocess-cancellation 0.1.6
    Uninstalling preprocess-cancellation-0.1.6:
      Successfully uninstalled preprocess-cancellation-0.1.6
  Attempting uninstall: pillow
    Found existing installation: Pillow 9.0.0
    Uninstalling Pillow-9.0.0:
      Successfully uninstalled Pillow-9.0.0
  Attempting uninstall: MarkupSafe
    Found existing installation: MarkupSafe 2.0.1
    Uninstalling MarkupSafe-2.0.1:
      Successfully uninstalled MarkupSafe-2.0.1
  Attempting uninstall: inotify-simple
    Found existing installation: inotify-simple 1.3.5
    Uninstalling inotify-simple-1.3.5:
      Successfully uninstalled inotify-simple-1.3.5
  Attempting uninstall: dbus-next
    Found existing installation: dbus-next 0.2.3
    Uninstalling dbus-next-0.2.3:
      Successfully uninstalled dbus-next-0.2.3
  Attempting uninstall: jinja2
    Found existing installation: Jinja2 3.0.3
    Uninstalling Jinja2-3.0.3:
      Successfully uninstalled Jinja2-3.0.3
Successfully installed MarkupSafe-2.0.1 dbus-next-0.2.3 distro-1.5.0 ifaddr-0.1.7 inotify-simple-1.3.5 jinja2-3.0.3 libnacl-1.7.2 lmdb-1.2.1 paho-mqtt-1.5.1 pillow-9.0.0 preprocess-cancellation-0.1.6 pycurl-7.44.1 pyserial-3.5 pyserial-asyncio-0.6 streaming-form-data-1.8.1 tornado-6.1 zeroconf-0.37.0

[m-sundman]

fwiw, I can't reproduce this issue with python 3.9 and pip 22.0.

This is still a problem with pip 22.0 and python 3.7. (No idea how to set it to use python 3.9 instead of 3.7.)

[Arksine]

I think I have a solution here.

FWIW, since you ran the install script with -f, it overwrote your service file, likely with the wrong config and log paths as you are on FluiddPi. See the Post below the one linked for the correct options to install-moonraker.sh.

[nickeeey]

idk if it helps you @Arksine but i (plus others from the Mainsail discord) got the same Problem

[Arksine]

I'm going to go ahead and close this issue to direct everyone to #352. I'll keep that one open for a bit, as I expect update failures to occur until pip releases a full fix.