Twitter Spaces Voice Client

[madjin]

Twitter Spaces Voice Client

Parity with Discord Voice Client, but on Twitter Space

• Must listen to and respond with voice
• Must be able to join a twitter space from a linked tweet
• Must be able to accept speaker role

Reward: $1000 USD in $ai16z + $1000 USD in $degenai as seen on https://ai16z.github.io/

[pukadev]

def use the openai real time api for a quick prototype to see if it catches on as a useful feature before spinning up your own multimodal TTS: we have worked with this openai recently.

https://github.com/openai/openai-realtime-api-beta

the key these though is making sure u control interruptions well at your llm responses and audio stream processing layer
(separate clean vocal segments, etc) ... there is a pace in real time voice that works best for the above api.. talking past each other., etc. and other background noise is a harder problem... it doesnt handle that very well unless u take over and dont turn on VAD for their api to do it for you.