docs: add encrypted fine-tuning documentation #887
Conversation
andrei-stoian-zama
force-pushed
the
docs/add_lora_docs
branch
from
2803dcf
to
e28dd06
Compare
docs/deep-learning/lora_training.md
Outdated
| Concrete ML requires a conversion step for the `peft` model, adding | ||
| FHE compatible layers. In this step the number of gradient accumulation steps | ||
| can also be set. For LORA it is common to accumulate gradients over | ||
| several gradient descent steps before updating weights. | ||
|
|
||
| <!--pytest-codeblocks:skip--> | ||
|
|
||
| `python` | ||
| gradient_accum_steps = 2 | ||
| lora_training = LoraTraining(peft_model, gradient_accum_steps) | ||
|
|
||
| ```` |
There was a problem hiding this comment.
I realize we don't handle that gradient accumulation properly. It's not going to be easy to fix we really need to integrate the training loop within the lora training. I will fix this in the second PR.
Currently, users need to handle the lora_training.run_optimizer(False|True) every gradient_accum_steps iteration. So basically we should probably limit gradient_accum_steps=1 for now. And not talk about it here.
| One fine-tuned, the LORA hybrid FHE model can perform inference only, through the | ||
| `model.inference_model` attribute of the hybrid FHE model. |
There was a problem hiding this comment.
So here I think we can simply re-use peft_model which should be updated. I will check quickly.
EDIT: I checked and indeed we can replace all the hybrid_model.... by lora_training or the peft_model depending on what we call making it much easier. And it's exactly the usage when using torch + peft fine-tuning.
| <!--pytest-codeblocks:skip--> | ||
|
|
||
| ```python | ||
| hybrid_model.model.inference_model(x) |
There was a problem hiding this comment.
same here we can just call peft_model
| hybrid_model.model.inference_model.disable_adapter_layers() | ||
| hybrid_model.model.inference_model(x) | ||
|
|
||
| # Re-enable the LORA weights | ||
| hybrid_model.model.inference_model.enable_adapter_layers() |
There was a problem hiding this comment.
same use peft_model everywhere.
Co-authored-by: Jordan Fréry <[email protected]>
Explains the LORA functionality added by #823