运行评估代码，ceval可以正常运行，但是，task切换cmmlu，报错

[situjunhao]

FlashAttention-2 is not installed, ignore this if you are not using FlashAttention.
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████| 7/7 [00:07<00:00, 1.14s/it]
09/28/2023 10:10:24 - INFO - llmtuner.tuner.core.adapter - Fine-tuning method: LoRA
09/28/2023 10:10:37 - INFO - llmtuner.tuner.core.adapter - Merged 1 model checkpoint(s).
09/28/2023 10:10:37 - INFO - llmtuner.tuner.core.adapter - Loaded fine-tuned model from checkpoint(s): ./outputs/sft
09/28/2023 10:10:40 - INFO - llmtuner.tuner.core.loader - trainable params: 0 || all params: 6243584000 || trainable%: 0.0000
Generating test split: 0 examples [00:00, ? examples/s] | 0/67 [00:00<?, ?it/s, 农学]
Processing subjects: 0%| | 0/67 [00:00<?, ?it/s, 农学]
Traceback (most recent call last):
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/builder.py", line 1693, in _prepare_split_single
example = self.info.features.encode_example(record) if self.info.features is not None else record
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/features/features.py", line 1852, in encode_example
return encode_nested_example(self, example)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/features/features.py", line 1229, in encode_nested_example
{
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/features/features.py", line 1229, in
{
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/utils/py_utils.py", line 323, in zip_dict
yield key, tuple(d[key] for d in dicts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/utils/py_utils.py", line 323, in
yield key, tuple(d[key] for d in dicts)
~^^^^^
KeyError: 'question'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/mnt/workspace/LLaMA-Efficient-Tuning/src/evaluate.py", line 188, in
fire.Fire(evaluate)
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/workspace/LLaMA-Efficient-Tuning/src/evaluate.py", line 136, in evaluate
dataset = load_dataset(os.path.join(dataset_dir, task), subject)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/load.py", line 2153, in load_dataset
builder_instance.download_and_prepare(
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/builder.py", line 954, in download_and_prepare
self._download_and_prepare(
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/builder.py", line 1717, in _download_and_prepare
super()._download_and_prepare(
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/builder.py", line 1049, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/builder.py", line 1555, in _prepare_split
for job_id, done, content in self._prepare_split_single(
File "/root/miniconda3/envs/llm/lib/python3.11/site-packages/datasets/builder.py", line 1712, in _prepare_split_single
raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.builder.DatasetGenerationError: An error occurred while generating the dataset

[situjunhao]

对应的执行脚本：
CUDA_VISIBLE_DEVICES=0 python src/evaluate.py
--model_name_or_path ../pretraining_model/chatglm2-6b
--finetuning_type lora
--checkpoint_dir ./outputs/sft
--template chatglm2
--task cmmlu
--split validation
--lang zh
--n_shot 5
--batch_size 4