Copy model before benchmark warmup runs (#145858)

Summary:
Fixes pytorch/pytorch#144772

The eager warmup runs causes the model to change state so that later when we export it, the model is different than when we export it directly out of box. For some reason exporting the model with the changed state causes issues but exporting the inital model is ok. This is the reason why the accuracy checks pass but the performance check fails when exporting.

X-link: pytorch/pytorch#145858
Approved by: https://github.com/desertfire

Reviewed By: ZainRizvi

Differential Revision: D68897720

fbshipit-source-id: 8b8f82c6986a1a6e7d92b8dabe3852cce6016d5c

userbenchmark/dynamo/dynamobench/common.py

@@ -3446,11 +3446,15 @@ def warmup(fn, model, example_inputs, mode, niters=5):
                 self.args.snapshot_memory, f"eager_{self.args.only}"
             ):
                 eager_latency, eager_peak_mem, _ = warmup(
-                    self.model_iter_fn, model, example_inputs, "eager"
+                    self.model_iter_fn, copy.deepcopy(model), example_inputs, "eager"
                 )
                 if self.args.use_warm_peak_memory:
                     _, eager_peak_mem, _ = warmup(
-                        self.model_iter_fn, model, example_inputs, "eager", niters=1
+                        self.model_iter_fn,
+                        copy.deepcopy(model),
+                        example_inputs,
+                        "eager",
+                        niters=1,
                     )
 
             if self.args.export_aot_inductor: