Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: note on intalling GPU toolkits for contributors #2035

Merged
merged 1 commit into from
May 3, 2024

Conversation

nuke-web3
Copy link
Contributor

@nuke-web3 nuke-web3 commented May 2, 2024

Toolchain installed on Ubuntu working with command cargo run -r --features cuda serve --model TabbyML/StarCoder-1B --device cuda.

For me I needed to set https://getdocs.org/Cmake/docs/3.21/envvar/cudacxx in your environment (~/.bashrc for me adding export CUDACXX=/usr/local/cuda/bin/nvcc after install of CUDA)

Closes #2033

@nuke-web3 nuke-web3 marked this pull request as draft May 2, 2024 23:04
@nuke-web3
Copy link
Contributor Author

I can run the service fine but get

DEBUG tabby::services::code: crates/tabby/src/services/code.rs:61: Source code index is not ready `Failed to open the directory: 'DoesNotExist("~/.tabby/index")'`

Is there some manual generation of that index needed on first startup not mentioned in CONTRIBUTING.md?

@wsxiaoys
Copy link
Member

wsxiaoys commented May 3, 2024

This is a debug warning, you can safely ignore it. To config and populate the index, use Git Providers manual in admin dashboard.

@nuke-web3
Copy link
Contributor Author

Got it working with admin panel http://localhost:8080/swagger-ui/#/v1/completion tested with "Try it out" to give something sensible 🎉

@wsxiaoys wsxiaoys changed the title Note on intalling GPU toolkits for contributors docs: note on intalling GPU toolkits for contributors May 3, 2024
@wsxiaoys wsxiaoys merged commit 24cebc1 into TabbyML:main May 3, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Nvidia GPU with CUDA support cargo run fails cmake with "CUDA not found"
2 participants