Distill trainer

[orenpereg]

1. Added Trainer for setfit distillation (file src/setfit/trainer_distill.py)
2. updated fewshot_distillation script to work with DistilSetFitTrainer

[lewtun]

Thanks for adding this nice feature @orenpereg 🔥 !

I've left a few comments / questions and would love to see some unit tests in a test_trainer_distillation.py file (you can take inspiration from the existing SetFitTrainer ones)

Could you also share a code snippet of how this trainer works end-to-end? I know we have that logic in the script, but it would be great to have a condensed version we can later add to the README :)

[orenpereg]

Hi @lewtun. Thanks for your thorough review and good comments🔥. I addressed all your comments and added tests under tests/test_trainer_distillation.py. can you pls take a look?
I also wrote a code snippet for the readme for using this trainer end-to-end:
Run distillation using trainer example.docx

[lewtun]

Thanks for iterating on this feature @orenpereg and adding some nice tests ❤️ !

I've left some final nits and then this should be good to merge :)

[lewtun]

Thanks for the last round of changes @orenpereg - this looks really nice 🔥

[orenpereg]

thanks Lewis :)