Update "getting started" using docker with a little more info

[johnwee1]

Problem

Hi, so I'm quite new to using tabbyAPI. i think the documentation for this is quite sparse, so just adding my two cents here.

I try using one of the Qwen2.5 models, running the docker instance of the server. I get this error

... tabbyapi-1 | ERROR: File "/usr/local/lib/python3.10/dist-packages/exllamav2/config.py", tabbyapi-1 | line 378, in prepare tabbyapi-1 | ERROR: raise ValueError(f" ## Could not find {prefix}.* in model") tabbyapi-1 | ERROR: ValueError: ## Could not find model.embed_tokens.* in model

which, after digging around, is because the prebuilt image isn't updated with the latest exllamav2 that supports qwen.

Solution

Just a simple line in the docs to specify that one might have to build the dockerfile first might solve headaches for a few users down the line, because that solved my issue.

Alternatives

No response

Explanation

Would help improve the documentation :)

Examples

No response

Additional context

No response

Acknowledgements

• I have looked for similar requests before submitting this one.
• I understand that the developers have lives and my issue will be answered when possible.
• I understand the developers of this program are human, and I will make my requests politely.