ValueError: too many file descriptors in select()

[lyoshenka]

was seeing this once in a while on windows + lots of blobs. Victor aware.

  File "c:\users\thoma\documents\lbry\lbrynet\extras\cli.py", line 287, in main
    loop.run_until_complete(daemon.stop_event.wait())
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 571, in run_until_complete
    self.run_forever()
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 539, in run_forever
    self._run_once()
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 1739, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\selectors.py", line 314, in _select
    r, w, x = select.select(r, w, w, timeout)
ValueError: too many file descriptors in select()

[tzarebczan]

Ran into this again while hosting content with about 300 files:

2019-07-31 07:39:38,418 INFO     lbry.blob_exchange.server:75: sent 466f72bc (2097152 bytes) to 80.229.160.127:60744
2019-07-31 07:39:42,384 INFO     lbry.blob_exchange.server:75: sent 2072ab33 (2097152 bytes) to 80.229.160.127:60751
Traceback (most recent call last):
  File "c:\users\thoma\documents\lbry\lbry\lbry\extras\cli.py", line 269, in run_daemon
    loop.run_forever()
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 539, in run_forever
    self._run_once()
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 1739, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\selectors.py", line 314, in _select
    r, w, x = select.select(r, w, w, timeout)
ValueError: too many file descriptors in select()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\thoma\Documents\lbry\lbry-venv\Scripts\lbrynet-script.py", line 11, in <module>
    load_entry_point('lbry', 'console_scripts', 'lbrynet')()
  File "c:\users\thoma\documents\lbry\lbry\lbry\extras\cli.py", line 294, in main
    run_daemon(args, conf)
  File "c:\users\thoma\documents\lbry\lbry\lbry\extras\cli.py", line 273, in run_daemon
    loop.run_until_complete(daemon.stop())
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 571, in run_until_complete
    self.run_forever()
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 539, in run_forever
    self._run_once()
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 1739, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\selectors.py", line 314, in _select
    r, w, x = select.select(r, w, w, timeout)
ValueError: too many file descriptors in select()```

[shyba]

Issue is: https://stackoverflow.com/questions/47675410/python-asyncio-aiohttp-valueerror-too-many-file-descriptors-in-select-on-win/47689451#47689451
but we cant change to proactor yet due lack of datagram endpoint support, which is solved on Python 3.8: https://docs.python.org/3.7/library/asyncio-eventloop.html#asyncio.loop.create_datagram_endpoint
-> https://docs.python.org/3.8/library/asyncio-eventloop.html#asyncio.loop.create_datagram_endpoint

[lyoshenka]

@shyba now that we're using python 3.8, can this be fixed? if you explain the fix, this would be a good issue for a newer contributor to try

[shyba]

I don't think we are using 3.8 officially. It may be compatible but afaik we still build and test 3.7

this is the search I just made and it looks like we don't set 3.8 anywhere yet: https://github.com/lbryio/lbry-sdk/search?q=python3.8

[belikor]

The RPC server does not work correctly with Python 3.8 or 3.9, which are now common in Ubuntu 20.04 and Arch. See #2769

So, until that issue is solved, LBRY is stuck in Python 3.7.

[belikor]

With the fixes proposed in #3357 and #3362, lbrynet seems to work fine with Python 3.8 and 3.9.