Update torchbench pin on PyTorch CI (#2584)

Summary:
I'm adding sam2 to TorchBench #2566, so I'm updating PyTorch CI to use latest TorchBench commit.

This goes together with pytorch/pytorch#145455.  The list of fixes includes:

* Adding `decoder_start_token_id` to HF `GenerationConfig`. This was Introduced by transformers 4.41.0 huggingface/transformers#30892
* Run `sam_fast` on A10G as it seems working correctly now

Pull Request resolved: #2584

Reviewed By: xuzhao9

Differential Revision: D68837682

Pulled By: huydhn

fbshipit-source-id: d9bda87384bac680b08fd2ea200ca7691c24a7e9

torchbenchmark/models/sam_fast/metadata.yaml

@@ -7,11 +7,7 @@ eval_nograd: true
 train_benchmark: false
 train_deterministic: false
 not_implemented:
-- device: cpu
-- device: cuda
-  test: example
-# eval test exceeds 300s time limit on A10G
-# https://github.com/pytorch/benchmark/actions/runs/8972426901/job/24640395990
-- device: NVIDIA A10G
-  test: eval
+  - device: cpu
+  - device: cuda
+    test: example
 skip_cuda_memory_leak: true

torchbenchmark/util/framework/huggingface/model_factory.py

@@ -165,6 +165,8 @@ def __init__(self, name, test, device, batch_size=None, extra_args=[]):
             do_sample=False,
             num_beams=1,
             use_cache=True,
+            # Introduced by transformers 4.41.0 https://github.com/huggingface/transformers/issues/30892
+            decoder_start_token_id=0,
         )
         self.model = GenerationWrapper(self.model, generation_config)
         self.example_inputs = (self.example_inputs["input_ids"],)