Questions about Nectar Datasets

[XinZhao0211]

Hi.
The paper mentions that the offline vanilla DPO is trained on the Nectar dataset. I have several questions about that.

1. How do you process the Nectar dataset? Nectar is the 7-wise comparison dataset which results in a total of 3.8M pairwise data. Do you use the whole dataset?
2. How do you set the beta in DPO training?

Thanks for your assistance.

[WeiXiongUST]

@hendrydong hi hanze, can you look into this?

[hendrydong]

Hi,

Pairs are randomly sampled for the whole data. Responses with higher scores are assigned as positive label; ones with lower scores are labeled as negative.

Beta is fixed as 0.1.

[XinZhao0211]

Hi,

Pairs are randomly sampled for the whole data. Responses with higher scores are assigned as positive label; ones with lower scores are labeled as negative.

Beta is fixed as 0.1.

How many pairs are randomly sampled for one prompt?
Could you please share the dataset?

[hendrydong]

You may have more details from Bo @bpucla .
Hi Bo, can you help to clarify the details of the offline baseline? Thanks