Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add template&support for c4ai-command-r/plus (tested) #3254

Conversation

marko1616
Copy link
Contributor

What does this PR do?

Add support for #3184

Before submitting

@marko1616 marko1616 changed the title Add template&support(Not tested) Add template&support(in test) Apr 12, 2024
@marko1616
Copy link
Contributor Author

Set param --use_fast_tokenizer true is in need

@marko1616
Copy link
Contributor Author

Template test pass using cli_demp.py
image

@marko1616
Copy link
Contributor Author

Sft test pass(Lora)

@marko1616 marko1616 changed the title Add template&support(in test) Add template&support(tested) Apr 12, 2024
@marko1616 marko1616 changed the title Add template&support(tested) Add template&support for c4ai-command-r/plus (tested) Apr 14, 2024
@marko1616
Copy link
Contributor Author

难绷捏,似乎挺难完美复刻huggingface上说的完整的template

@hiyouga hiyouga self-requested a review April 15, 2024 14:58
@hiyouga hiyouga merged commit 7a8ae3f into hiyouga:main Apr 15, 2024
1 check passed
@hiyouga hiyouga added the solved This problem has been already solved label Apr 15, 2024
@hiyouga
Copy link
Owner

hiyouga commented Apr 15, 2024

Some necessary changes have been made and this pr can be safely merged

@hiyouga
Copy link
Owner

hiyouga commented Apr 15, 2024

Set param --use_fast_tokenizer true is in need

fixed in e0dbac2

@marko1616 marko1616 deleted the feature/Add-support-for-CohereForAI/c4ai-command-r-plus branch August 22, 2024 03:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
solved This problem has been already solved
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants