playai: enable streaming TTS #1340
Conversation
🦋 Changeset detectedLatest commit: 66c72e1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Relocated client creation to the _run method to handle occasional WebSocket connection drops by creating a new client. |
| self._client = PlayHTAsyncClient( | ||
| user_id=self._user_id, | ||
| api_key=self._api_key, | ||
| ) |
There was a problem hiding this comment.
Do we want to recreate a new client for every request?
There was a problem hiding this comment.
Yeah, it's a workaround for now. WebSocket connection drops from playht server sometimes and it's not handled in pyht sdk.
There was a problem hiding this comment.
Hey guys, this is causing latency problems for customers--there is a significant overhead around creating the connection on every request.
There was a problem hiding this comment.
Hi, The workaround was introduced to mitigate occasional connection drops from the PlayHT server, as the current pyht SDK doesn't seem to handle these drops robustly.
packages/pyht/async_client.py", line 471, in _tts_ws
await self._ws.send(json.dumps(json_data))
File "/Users/jayesh/dev/livekit/agents/venv/lib/python3.10/site-packages/websockets/asyncio/connection.py", line 403, in send
async with self.send_context():
File "/opt/homebrew/Cellar/[email protected]/3.10.16/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 199, in __aenter__
return await anext(self.gen)
File "/Users/jayesh/dev/livekit/agents/venv/lib/python3.10/site-packages/websockets/asyncio/connection.py", line 873, in send_context
raise self.protocol.close_exc from original_exc
websockets.exceptions.ConnectionClosedError: no close frame received or sent
Could the SDK be updated to include automatic reconnection? This would greatly help reduce overhead
Let me know your thoughts!
There was a problem hiding this comment.
Can you verify for me the pyht version that you have installed (even if a newer one is in your requirements.txt, maybe it has not actually been upgraded)?
Currently the ws.send() call is in a try-except that should re-establish the connection if there is an exception. Could be it is still dropping this somehow! But curious if upgrading to the latest pyht will help.
There was a problem hiding this comment.
I think I figured out how this can still occur even in the latest pyht.
It is a little hard to test it as I have not been able to reproduce the disconnect, but I have a PR here and will try to have it tested and released tomorrow so you can remove the per-request reconnection.
No description provided.