Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请在llamafactory-cli api中支持设置root_path #5307

Closed
1 task done
shizidushu opened this issue Aug 29, 2024 · 1 comment
Closed
1 task done

请在llamafactory-cli api中支持设置root_path #5307

shizidushu opened this issue Aug 29, 2024 · 1 comment
Labels
solved This problem has been already solved

Comments

@shizidushu
Copy link

Reminder

  • I have read the README and searched the existing issues.

System Info

  • llamafactory version: 0.8.4.dev0
  • Platform: Linux-5.4.0-186-generic-x86_64-with-glibc2.35
  • Python version: 3.10.14
  • PyTorch version: 2.4.0+cu121 (GPU)
  • Transformers version: 4.43.4
  • Datasets version: 2.20.0
  • Accelerate version: 0.32.0
  • PEFT version: 0.12.0
  • TRL version: 0.9.6
  • vLLM version: 0.5.5

Reproduction

按照文章 https://zhuanlan.zhihu.com/p/695287607 直到启动API Server

CUDA_VISIBLE_DEVICES=0 API_PORT=8000 llamafactory-cli api \
    --model_name_or_path /root/workspace/models-modelscope/Meta-Llama-3-8B-Instruct \
    --adapter_name_or_path ./saves/LLaMA3-8B/lora/sft \
    --template llama3 \
    --finetuning_type lora

由于我的环境需要配置root_path,所以服务无法正常运行,root_path 文档(FastAPI)请参考: https://fastapi.tiangolo.com/zh/advanced/behind-a-proxy/

Expected behavior

No response

Others

No response

@github-actions github-actions bot added the pending This problem is yet to be addressed label Aug 29, 2024
@hiyouga
Copy link
Owner

hiyouga commented Aug 29, 2024

use FASTAPI_ROOT_PATH

@hiyouga hiyouga added solved This problem has been already solved and removed pending This problem is yet to be addressed labels Aug 29, 2024
yuwangnexusera pushed a commit to yuwangnexusera/LLaMA-Factory that referenced this issue Sep 5, 2024
yuwangnexusera pushed a commit to yuwangnexusera/LLaMA-Factory that referenced this issue Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
solved This problem has been already solved
Projects
None yet
Development

No branches or pull requests

2 participants