Add various social bias tasks

[oskarvanderwal]

This PR implements various popular benchmarks for evaluating LMs for social biases. I also aim to have these validated where possible: e.g., by comparing with existing implementations or results, or at least have a second person indicate it makes sense. @StellaAthena suggested to get some help in validating the implementations.

Benchmark	Implemented	Validated	Used by e.g.
Simple Cooccurrence Bias	This PR		GPT-3, PALM, Gopher
Winogender	This PR		GPT-3, LLaMA, PALM, Chinchilla, Gemma
BBQ	This PR		BIG-bench, HELM, PaLM 2, Gemma
StereoSet	skipping		Jurassic-1, OPT, Galactica
CrowS-Pairs	already implemented		LLaMA-65B, BLOOM, OPT, Pythia, Galactica, StarCoder, Gemma
RealToxicityPrompts	already implemented		LLaMA, PALM, Gopher, OPT, PaLM 2, Gemma

Winogender

This implementation follows the description from the paper "Language Models are Few-Shot Learners".
However, for validation, I compare my results with the results reported in the LLaMA paper, who should have the same implementation.

Unfortunately, their results (reported in Table 13 of the LLaMA paper) are not reproduced by my implementation:

Subset	7B
All	66.0
her/her/she	65.0
his/him/he	60.8
their/them/someone	72.1
her/her/she (gotcha)	64.2
his/him/he (gotcha)	55.0

For the 7B LLaMA model (using the llama-cpp-python server with mps), I got the following for different prompts:

Prompt 1: "{{sentence}} ‘{{pronoun.capitalize()}}’ refers to the"

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
winogender	Yaml	none	0	acc	0.6208	±	0.0181
winogender_female	Yaml	none	0	acc	0.6208	±	0.0314
winogender_male	Yaml	none	0	acc	0.6042	±	0.0316
winogender_neutral	Yaml	none	0	acc	0.6375	±	0.0311
winogender_gotcha_female	Yaml	none	0	acc	0.6000	±	0.0449
winogender_gotcha_male	Yaml	none	0	acc	0.5417	±	0.0457

Prompt 2: "{{sentence}}\n\n‘{{pronoun.capitalize()}}’ refers to the"

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
- winogender_all	1	none	0	acc	0.5861	±	0.0184
- winogender_female	1	none	0	acc	0.5792	±	0.0319
- winogender_male	1	none	0	acc	0.5792	±	0.0319
- winogender_neutral	1	none	0	acc	0.6000	±	0.0317

Prompt 1 + description: "Answer these questions:\n\n"

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
- winogender_all	1	none	0	acc	0.5958	±	0.0183
- winogender_female	1	none	0	acc	0.6000	±	0.0317
- winogender_male	1	none	0	acc	0.5708	±	0.0320
- winogender_neutral	1	none	0	acc	0.6167	±	0.0314

BBQ

• Compare with HELM implementation

StereoSet

• [ ] Finish implementation
• [ ] Compare GPT-2 results with StereoSet leaderboard

[CLAassistant]

All committers have signed the CLA.

[oskarvanderwal]

BBQ required me to implement custom metrics. Interestingly, everything works when running each subset of BBQ individually, but I run into a problem when running instead the bbq group:
TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int'
This seems to happen when computing the var_score in evaluator.py, where stderr is a string.

This is a minor issue for me, but worth diving into?

[oskarvanderwal]

For now, I decided against including StereoSet.

[StellaAthena]

Does the BBQ implementation replicate any papers @oskarvanderwal?

And if the Winogender implementation matched the Winogrande one (from this repo which should be the same as the GPT-3 paper) according to --log_samples we're fine.

[oskarvanderwal]

@StellaAthena

• Winogender does currently not replicate the results reported in the LLaMA v1 paper. @lintangsutawika suggested trying different prompts to see if one variation does agree. I have no way to verify with the "original" implementation from GPT-3. EDIT: I'll check with the Winogrande implementation!
• For BBQ, I haven't run the full suite yet, and hoped @lintangsutawika could help me out here. HELM reports BBQ results for a bunch of larger models here; I think GPT-J (6B) would be a good candidate. What makes it difficult to completely validate it with HELM, is that I couldn't find which 5 examples they used for their few-shot evaluation. I did take the HELM implementation as basis for my implementation, but wasn't able to do so 1-to-1. If I understand HELM correctly, it does not evaluate multiple choice in the same way as eval-harness. The dataset (my copy here) required quite some post-processing as well, which is another potential source of deviations.

[StellaAthena]

@oskarvanderwal I don't see anything in the BigScience fork of this repo or in the BLOOM paper about BBQ.

The BBQ GitHub repo says the following:

For testing Unified QA, we used an off-the-shelf model. String formatting for inference was created by concatenating the following fields from the data files:

RACE-style-format: question + \n + '(a)' + ans_0 + '(b)' + ans_1 + '(c)' + ans2 + \n + context

ARC-style-format: context + question + \n + '(a)' + ans_0 + '(b)' + ans_1 + '(c)' + ans2

The BigBench version appears to be: 'Q: ' + context + question + '\n  choice: ' + ans_0 + '\n  choice: ' + ans_1 + '\n  choice: ' + ans_3 + '\nA: '

I haven't figured out how to read HELM prompts yet... should get to learning that :/

@Alicia-Parrish is there a way that you would like us to format the BBQ evaluation for generative models?

[Alicia-Parrish]

For generative models, you can follow the method that we used for evaluation in the PaLM2 technical report -- see Appendix D.6. The input format was just "{{context}}.\n\nQ: {{question}}\nA:" (so, none of the multiple choice answer options are shown). Then the response can be evaluated by just using string search for each of the two individuals introduced in the context (this is easier when you use the metadata fields than the answer options). This worked pretty well and only required manual coding of 14% of examples.

If you prefer evaluating still using the multiple choice format, I would just recommend using a few different formats that the model has seen before, to make sure that model behavior isn't super different across format options.

[oskarvanderwal]

@StellaAthena re: mentioning BLOOM using BBQ, you're right in that BLOOM didn't explicitly evaluate on BBQ. They did however evaluate on HELM and report their results on the bias category in Figure 10. I wrongly assumed this targeted the BBQ dataset, but looking at the HELM paper more carefully I don't think this is the case. I'll remove the mention from the column.

About implementing BBQ: do you think we should follow the methodology described in the PaLM2 technical report (which I think is similar to HELM)? Or do you prefer the BIG-bench approach or something more closely to how eval-harness currently evaluates multiple choice?

[StellaAthena]

@Alicia-Parrish Thanks!

@oskarvanderwal Let's support the standard Q-A formatting that we typically use and also the palm2 formatting.

I specifically dislike the BigBench approach and think we should never use it except for the sake of replication. We can create a metaprompt template for it.

[justinphan3110cais]

@oskarvanderwal , Hi Oskar, quick question, why did you decide against stereoset?

[oskarvanderwal]

@justinphan3110cais, mainly because of time constraints on my end. And since I am mainly interested in evaluating autoregressive models, the implementation of StereoSet would also become more complicated. (It was mainly designed for masked language models, although the authors do suggest an adaptation in their paper). And last of all, StereoSet and CrowS-Pairs are already fairly similar in their approach to measuring bias.

Just to clarify: my main focus is on testing the validity/reliability of said bias benchmarks, more so than measuring bias in LMs.

[notrichardren]

Hi! How is progress on adding these evaluations to lm-eval-harness?