Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] Preprocess voice in the client and only send data when spoken #15

Open
HarikalarKutusu opened this issue Apr 5, 2022 · 0 comments
Labels
backend Server related enhancement New feature or request frontend Client related

Comments

@HarikalarKutusu
Copy link
Owner

HarikalarKutusu commented Apr 5, 2022

Currently, all sounds are streamed continuously, which also hogs the server. In a normal chess play there are 30-50 moves * 2 per game in 20-120 minutes. Assuming 30 minutes/game with 40 moves/player and 4 secs per command, that would mean less than 10%.

If we can implement this:

  • Any silence, background noise etc will be filtered out.
  • Only relevant data will be sent, so less communication on both sides
  • We can use the same server for multiple connections by queuing incoming packets. In this case, a server can become a "language server" with (say) 5-10 connection/inference capacity (which can be adaptive). This is as indicated in [FR] Increase server resources - the scaling problem #4 , third option.

This can be achieved by measuring the sound level/energy with pre-buffering.

One downside to this is: The client will be processing audio continuously, which can be bad for future mobile users in terms of battery usage.

@HarikalarKutusu HarikalarKutusu added enhancement New feature or request backend Server related frontend Client related labels Apr 5, 2022
@HarikalarKutusu HarikalarKutusu moved this from Todo to Backlog in 3d-voice-chess Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Server related enhancement New feature or request frontend Client related
Projects
Status: Backlog
Development

No branches or pull requests

1 participant