Unable to connect :

[Sigmun]

When I tried to connect to my nextcloud server on an OVH Web Host, I get the following error:

>>> client.principal()
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py in _update_chunk_length(self)
    696         try:
--> 697             self.chunk_left = int(line, 16)
    698         except ValueError:

ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

InvalidChunkLength                        Traceback (most recent call last)
C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py in _error_catcher(self)
    437             try:
--> 438                 yield
    439

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py in read_chunked(self, amt, decode_content)
    763             while True:
--> 764                 self._update_chunk_length()
    765                 if self.chunk_left == 0:

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py in _update_chunk_length(self)
    700             self.close()
--> 701             raise InvalidChunkLength(self, line)
    702

InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)

During handling of the above exception, another exception occurred:

ProtocolError                             Traceback (most recent call last)
C:\ProgramData\Anaconda3\lib\site-packages\requests\models.py in generate()
    757                 try:
--> 758                     for chunk in self.raw.stream(chunk_size, decode_content=True):
    759                         yield chunk

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py in stream(self, amt, decode_content)
    571         if self.chunked and self.supports_chunked_reads():
--> 572             for line in self.read_chunked(amt, decode_content=decode_content):
    573                 yield line

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py in read_chunked(self, amt, decode_content)
    792             if self._original_response:
--> 793                 self._original_response.close()
    794

C:\ProgramData\Anaconda3\lib\contextlib.py in __exit__(self, typ, value, traceback)
    136             try:
--> 137                 self.gen.throw(typ, value, traceback)
    138             except StopIteration as exc:

C:\ProgramData\Anaconda3\lib\site-packages\urllib3\response.py in _error_catcher(self)
    454                 # This includes IncompleteRead.
--> 455                 raise ProtocolError("Connection broken: %r" % e, e)
    456

ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

ChunkedEncodingError                      Traceback (most recent call last)
<ipython-input-8-a6f62e4f6479> in <module>
----> 1 client.principal()

C:\ProgramData\Anaconda3\lib\site-packages\caldav\davclient.py in principal(self, *largs, **kwargs)
    337         """
    338         if not self._principal:
--> 339             self._principal = Principal(client=self, *largs, **kwargs)
    340         return self._principal
    341

C:\ProgramData\Anaconda3\lib\site-packages\caldav\objects.py in __init__(self, client, url)
    386         if url is None:
    387             self.url = self.client.url
--> 388             cup = self.get_property(dav.CurrentUserPrincipal())
    389             self.url = self.client.url.join(
    390                 URL.objectify(cup))

C:\ProgramData\Anaconda3\lib\site-packages\caldav\objects.py in get_property(self, prop, use_cached, **passthrough)
    168             if prop.tag in self.props:
    169                 return self.props[prop.tag]
--> 170         foo = self.get_properties([prop], **passthrough)
    171         return foo.get(prop.tag, None)
    172

C:\ProgramData\Anaconda3\lib\site-packages\caldav\objects.py in get_properties(self, props, depth, parse_response_xml, parse_props)
    191         """
    192         rc = None
--> 193         response = self._query_properties(props, depth)
    194         if not parse_response_xml:
    195             return response

C:\ProgramData\Anaconda3\lib\site-packages\caldav\objects.py in _query_properties(self, props, depth)
    135             root = dav.Propfind() + prop
    136
--> 137         return self._query(root, depth)
    138
    139     def _query(self, root=None, depth=0, query_method='propfind', url=None,

C:\ProgramData\Anaconda3\lib\site-packages\caldav\objects.py in _query(self, root, depth, query_method, url, expected_return_value)
    153         if url is None:
    154             url = self.url
--> 155         ret = getattr(self.client, query_method)(
    156             url, body, depth)
    157         if ret.status == 404:

C:\ProgramData\Anaconda3\lib\site-packages\caldav\davclient.py in propfind(self, url, props, depth)
    372          * DAVResponse
    373         """
--> 374         return self.request(url or self.url, "PROPFIND", props,
    375                             {'Depth': str(depth)})
    376

C:\ProgramData\Anaconda3\lib\site-packages\caldav\davclient.py in request(self, url, method, body, headers)
    493             auth = self.auth
    494
--> 495         r = self.session.request(
    496             method, url, data=to_wire(body),
    497             headers=combined_headers, proxies=proxies, auth=auth,

C:\ProgramData\Anaconda3\lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    540         }
    541         send_kwargs.update(settings)
--> 542         resp = self.send(prep, **send_kwargs)
    543
    544         return resp

C:\ProgramData\Anaconda3\lib\site-packages\requests\sessions.py in send(self, request, **kwargs)
    695
    696         if not stream:
--> 697             r.content
    698
    699         return r

C:\ProgramData\Anaconda3\lib\site-packages\requests\models.py in content(self)
    834                 self._content = None
    835             else:
--> 836                 self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
    837
    838         self._content_consumed = True

C:\ProgramData\Anaconda3\lib\site-packages\requests\models.py in generate()
    759                         yield chunk
    760                 except ProtocolError as e:
--> 761                     raise ChunkedEncodingError(e)
    762                 except DecodeError as e:
    763                     raise ContentDecodingError(e)

ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

This calendar is correctly set up on thunderbird or DavX5.

Any idea? Is it my server configuration? or OVH?

[tobixen]

I've never seen such problems before - this is the requests library, breaking while reading the response from the server. So it doesn't have anything to do with the caldav library, per se.

I've tried googling a bit, the only thing I found on that error message was https://stackoverflow.com/questions/68584385/invalid-chunk-length-pull-zip-file-from-url-to-python ... weird :-(

You can try in C:\ProgramData\Anaconda3\lib\site-packages\caldav\objects.py just before line 155 to add print(url) just to see if what kind of URL you get there.

I did run the caldav test suite towards NextCloud at https://e.email, and I did succed with that.

[Sigmun]

I get an error on the print command:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/conda/envs/39/lib/python3.9/site-packages/urllib3/response.py in _update_chunk_length(self)
    696         try:
--> 697             self.chunk_left = int(line, 16)
    698         except ValueError:

ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

InvalidChunkLength                        Traceback (most recent call last)
/conda/envs/39/lib/python3.9/site-packages/urllib3/response.py in _error_catcher(self)
    437             try:
--> 438                 yield
    439 

/conda/envs/39/lib/python3.9/site-packages/urllib3/response.py in read_chunked(self, amt, decode_content)
    763             while True:
--> 764                 self._update_chunk_length()
    765                 if self.chunk_left == 0:

/conda/envs/39/lib/python3.9/site-packages/urllib3/response.py in _update_chunk_length(self)
    700             self.close()
--> 701             raise InvalidChunkLength(self, line)
    702 

InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)

During handling of the above exception, another exception occurred:

ProtocolError                             Traceback (most recent call last)
/conda/envs/39/lib/python3.9/site-packages/requests/models.py in generate()
    757                 try:
--> 758                     for chunk in self.raw.stream(chunk_size, decode_content=True):
    759                         yield chunk

/conda/envs/39/lib/python3.9/site-packages/urllib3/response.py in stream(self, amt, decode_content)
    571         if self.chunked and self.supports_chunked_reads():
--> 572             for line in self.read_chunked(amt, decode_content=decode_content):
    573                 yield line

/conda/envs/39/lib/python3.9/site-packages/urllib3/response.py in read_chunked(self, amt, decode_content)
    792             if self._original_response:
--> 793                 self._original_response.close()
    794 

/conda/envs/39/lib/python3.9/contextlib.py in __exit__(self, type, value, traceback)
    134             try:
--> 135                 self.gen.throw(type, value, traceback)
    136             except StopIteration as exc:

/conda/envs/39/lib/python3.9/site-packages/urllib3/response.py in _error_catcher(self)
    454                 # This includes IncompleteRead.
--> 455                 raise ProtocolError("Connection broken: %r" % e, e)
    456 

ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

During handling of the above exception, another exception occurred:

ChunkedEncodingError                      Traceback (most recent call last)
/ipykernel_26517/541589744.py in <module>
      4 cal_name = "Personnel"
      5 client = caldav.DAVClient(username="xxx", password="xxx", url=f"{caldav_serveur}")
----> 6 cal_lyra = client.principal().calendar(name=cal_name)
      7 print(cal_lyra)
      8 lyra_events = cal_lyra.objects(load_objects=True)

/conda/envs/39/lib/python3.9/site-packages/caldav/davclient.py in principal(self, *largs, **kwargs)
    337         """
    338         if not self._principal:
--> 339             self._principal = Principal(client=self, *largs, **kwargs)
    340         return self._principal
    341 

/conda/envs/39/lib/python3.9/site-packages/caldav/objects.py in __init__(self, client, url)
    386         self._calendar_home_set = None
    387 
--> 388         if url is None:
    389             self.url = self.client.url
    390             cup = self.get_property(dav.CurrentUserPrincipal())

/conda/envs/39/lib/python3.9/site-packages/caldav/objects.py in get_property(self, prop, use_cached, **passthrough)
    168         ## TODO: use_cached should probably be true
    169         if use_cached:
--> 170             if prop.tag in self.props:
    171                 return self.props[prop.tag]
    172         foo = self.get_properties([prop], **passthrough)

/conda/envs/39/lib/python3.9/site-packages/caldav/objects.py in get_properties(self, props, depth, parse_response_xml, parse_props)
    191          * {proptag: value, ...}
    192 
--> 193         """
    194         rc = None
    195         response = self._query_properties(props, depth)

/conda/envs/39/lib/python3.9/site-packages/caldav/objects.py in _query_properties(self, props, depth)
    135             root = dav.Propfind() + prop
    136 
--> 137         return self._query(root, depth)
    138 
    139     def _query(self, root=None, depth=0, query_method='propfind', url=None,

/conda/envs/39/lib/python3.9/site-packages/caldav/objects.py in _query(self, root, depth, query_method, url, expected_return_value)
    153         if url is None:
    154             url = self.url
--> 155         print("DEBUG:url:")
    156         print(url)
    157         ret = getattr(self.client, query_method)(

/conda/envs/39/lib/python3.9/site-packages/caldav/davclient.py in propfind(self, url, props, depth)
    372          * DAVResponse
    373         """
--> 374         return self.request(url or self.url, "PROPFIND", props,
    375                             {'Depth': str(depth)})
    376 

/conda/envs/39/lib/python3.9/site-packages/caldav/davclient.py in request(self, url, method, body, headers)
    493             auth = self.auth
    494 
--> 495         r = self.session.request(
    496             method, url, data=to_wire(body),
    497             headers=combined_headers, proxies=proxies, auth=auth,

/conda/envs/39/lib/python3.9/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    540         }
    541         send_kwargs.update(settings)
--> 542         resp = self.send(prep, **send_kwargs)
    543 
    544         return resp

/conda/envs/39/lib/python3.9/site-packages/requests/sessions.py in send(self, request, **kwargs)
    695 
    696         if not stream:
--> 697             r.content
    698 
    699         return r

/conda/envs/39/lib/python3.9/site-packages/requests/models.py in content(self)
    834                 self._content = None
    835             else:
--> 836                 self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
    837 
    838         self._content_consumed = True

/conda/envs/39/lib/python3.9/site-packages/requests/models.py in generate()
    759                         yield chunk
    760                 except ProtocolError as e:
--> 761                     raise ChunkedEncodingError(e)
    762                 except DecodeError as e:
    763                     raise ContentDecodingError(e)

ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))

Would you like me to create you an account on my NC server?

[Sigmun]

Still not working with 0.8.1

[tobixen]

Would you like me to create you an account on my NC server?

Yes please, I would love to run tests towards it. I've tested NextCloud at try.nextcloud.com and tests passes (except that sync-tokens and expandation of recurring events/todos doesn't work). I tried nextcloud at e.email / cloud.global, and got the same results a couple of weeks ago, but arbitrary errors when I tried some few days ago.

Here are a couple of suggestions on how to set a password:

• Set some password, encrypt it with my gpg key from https://calendarfest.tobixen.no/B14605A5.txt
• Fetch the password that I've posted at https://pwpush.redpill-linpro.com/en/p/tfhpl1d7tt90jfkj/r

You may eventually send credentials by email, i.e. [email protected]

[tobixen]

... or just use the NextCloud functionality for setting a new password, great :-)

[tobixen]

I also get those InvalidChunkLength when running the tests. Very weird indeed. Could it have something to do with tcp segment size possibly?

I will do research into this, but cannot promise any timeline.

[tobixen]

Unfortunately so far I can only report .... "I understand nothing". I can most likely create some workaround for this problem, but it would be better to understand what's going on. So I will do more research.

>>> sys.path.insert(0, '.') ; import caldav ; from tests.conf import client
>>> c=client()
>>> c.principal()
Traceback (most recent call last):
(...)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)
>>> c.principal()
Traceback (most recent call last):
(...)
urllib3.exceptions.InvalidChunkLength: InvalidChunkLength(got length b'', 0 bytes read)
>>> c.propfind(c.url)
<caldav.davclient.DAVResponse object at 0x7f32aade2970>
>>> _.raw
b'<?xml version="1.0"?>\n<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns"><d:response><d:href>/remote.php/dav/</d:href><d:propstat><d:prop><d:resourcetype><d:collection/></d:resourcetype></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response></d:multistatus>\n'
>>> c.principal()
Principal(https://cloud.xxx.fr/remote.php/dav/principals/users/tobixen/)

[tobixen]

Ok, I'm a bit smarter now.

There are two different ways of doing authentication over http, it's "digest" and "basic auth". The current code attempts to do a digest first, and if it fails with a 401, it falls back to basic auth. This caldav server gives a 401 when doing i.e. a PROPFIND with the digest method and an empty body, but it throws some spectacular error when doing a PROPFIND with a body and the digest method.

There exists a workaround that works right now, the DAVClient class may take an auth object:

>>> sys.path.insert(0, '.') ; import caldav ; from tests.conf import client ; c=client()
>>> import requests
>>> c = caldav.DAVClient(url=c.url, auth=requests.auth.HTTPBasicAuth(c.username, c.password))
>>> c.principal()
Principal(https://cloud.xxxx.fr/remote.php/dav/principals/users/tobixen/)

I still believe this is more a server error than a client error, but I will read up a bit on the http standard to see if there are any better methods to discover weather the server expects digest or basic auth.

[tobixen]

When doing a simple proppatch towards the caldav URL without any authentication information, it shows this:

(...)
WWW-Authenticate: Basic realm="Nextcloud"
(...)
401 Unauthorized
(...)

The "basic" above indicates that "basic auth" should be used rather than "digest". Perhaps the server is terminating the connection before the request is completed when a body is sent rogether with the wrong auth type.

I find it strange that the "elegant and simple HTTP library for Python, built for human beings" can't accept a simple username and password and choose one auth over the other by itself - but anyway, I will build proper auth type discovery into the library, hopefully this will solve the issue with this server (and perhaps it will solve other weird issues as well). While I'm at it, maybe I'll have a stab at #119

[Sigmun]

It is working using 0.8.2. Thanks

[Sigmun]

One more thing perhaps, Can you hide the NC url in your outputs above?

[tobixen]

Sorry, I was a bit uncertain if that was a public URL or not

[Sigmun]

No problem, thanks for the fix!

[tobixen]

Since the previous code worked OK with other nextcloud instances, I think the real culprit here was in a layer above nextcloud (apache or nginx setup).

Now that I'm trying to clean away the old code, radicale breaks. It has sort of the opposite behaviour. Without the body and no auth given, it will give a 207 instead of a 401.