Unexpected keyword argument 'beta' in DPOTrainer initialization

[YijuGuo]

When running the training code in the rlhflow environment, I encountered a TypeError with the message: DPOTrainer.init() got an unexpected keyword argument 'beta'. It seems like there might be an issue with the compatibility of the arguments in the DPOTrainer initialization.

Could you provide information on the specific version of trl used in the rlhflow environment to investigate this issue further?

Thank you for your support and guidance. Your assistance in clarifying this matter is greatly appreciated.

Warm regards,
Yiju Guo