Questions about the implementation of the loss function

[Weiting-Gao]

Thanks for sharing the code! I read the paper and also checked the code. I’m currently trying to adopt Diffmask to another dataset and have some questions regarding the code:

1. What is alpha (defined in sentiment_classification_sst_diffmask.py BertSentimentClassificationSSTDiffMask), is that Lagrangian multiplier mentioned in Eq(3) in the paper?

2. In SentimentClassificationSSTDiffMask, What is the expected_L0 in loss_g, why expected_L0 is negative? The negative value of expected_L0 makes loss_g negative. Is that correct?

3. I also don’t understand log_expected_L0() function in distributions.py. Can I find an explanation for this in the paper?

4. During the training step, you also calculate l0 (l0 = (expected_L0.exp() * mask).sum(-1) / mask.sum(-1)), what is this for, is this used for training?

Again, thanks for the wonderful work. Look forward to your reply!