Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Please, integrate PiSSA method for LoRA tuning #3196

Closed
1 task done
gotzmann opened this issue Apr 9, 2024 · 2 comments · Fixed by #4307
Closed
1 task done

Please, integrate PiSSA method for LoRA tuning #3196

gotzmann opened this issue Apr 9, 2024 · 2 comments · Fixed by #4307
Labels
duplicate This issue or pull request already exists

Comments

@gotzmann
Copy link

gotzmann commented Apr 9, 2024

Reminder

  • I have read the README and searched the existing issues.

Reproduction

python train

Expected behavior

No response

System Info

No response

Others

https://github.com/GraphPKU/PiSSA

peft_config = LoraConfig(
    ...
    init_lora_weights='pissa', # PiSSA initialization
    ...
)
@gotzmann
Copy link
Author

gotzmann commented Apr 9, 2024

Oh, it's not so easy, there non-standard PEFT needed

huggingface/peft@main...fxmeng:peft:main

@hiyouga hiyouga added the pending This problem is yet to be addressed label Apr 9, 2024
@hiyouga
Copy link
Owner

hiyouga commented Apr 17, 2024

Duplicate of #3162

@hiyouga hiyouga marked this as a duplicate of #3162 Apr 17, 2024
@hiyouga hiyouga added duplicate This issue or pull request already exists and removed pending This problem is yet to be addressed labels Apr 17, 2024
@hiyouga hiyouga closed this as completed Apr 17, 2024
@hiyouga hiyouga mentioned this issue Jun 15, 2024
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate This issue or pull request already exists
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support pissa hiyouga/LLaMA-Factory
2 participants
@gotzmann @hiyouga