Error loading script: trt.py, ModuleNotFoundError: No module named 'tensorrt_bindings' #27

rlenci99 · 2023-10-18T01:31:27Z

Hello, i have fresh installed using installation method provided and the extension will install, but the tab for TensorRT will not show.

*** Error loading script: trt.py
Traceback (most recent call last):
File "E:\Stable diffusion\SD\webui\modules\scripts.py", line 382, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "E:\Stable diffusion\SD\webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in
import ui_trt
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in
from exporter import export_onnx, export_trt
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in
from utilities import Engine
File "E:\Stable diffusion\SD\webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in
import tensorrt as trt
File "E:\Stable diffusion\SD\webui\venv\lib\site-packages\tensorrt_init.py", line 18, in
from tensorrt_bindings import *
ModuleNotFoundError: No module named 'tensorrt_bindings'

racerx2oo3 · 2023-10-18T03:31:32Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists
Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

Run webui.bat - this should rebuild the virtual environment venv
When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

enter: venv\Scripts\activate.bat
the command line should now have (venv) shown at the beginning.
enter the following commands:
- python.exe -m pip install --upgrade pip
- python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
- python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
- python -m pip uninstall -y nvidia-cudnn-cu11
- venv\Scripts\deactivate.bat
webui.bat
Install the TensorRT extension using the Install from URL option
Once installed, go to the Extensions >> Installed tab and Apply and Restart

rlenci99 · 2023-10-18T04:25:44Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

Run webui.bat - this should rebuild the virtual environment venv

When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

enter: venv\Scripts\activate.bat

the command line should now have (venv) shown at the beginning.

enter the following commands:

python.exe -m pip install --upgrade pip

python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

python -m pip uninstall -y nvidia-cudnn-cu11

venv\Scripts\deactivate.bat

webui.bat

Install the TensorRT extension using the Install from URL option

Once installed, go to the Extensions >> Installed tab and Apply and Restart

Thanks for the detailed steps, much appreciated. It worked!!

NicoBerlo · 2023-10-18T13:56:19Z

What appears to have worked for others.
(...)
* Once installed, go to the Extensions >> Installed tab and Apply and Restart

Hi !

I've managed to get it to work with v1-5-pruned-emaonly.safetensors [6ce0161689], but when I try with sd_xl_base_1.0.safetensors [31e35c80fc], I get the following error (I created a profile for the sd_xl_base_1.0 tensors which I selected in the UI, and I'm using a shape that I generated in the profile) :
*** Error completing request *** Arguments: ('task(l8v3an5pj39id83)', 'one robot on a bike', '', [], 80, 'Euler a', 1, 1, 7, 768, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000001EBA30067D0>, 0, False, '', 0.8, 194968604, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}

` (...)
File "C:\Users\nicolas\ai\sd.webui\webui\extensions-builtin\Lora\networks.py", line 486, in network_MultiheadAttention_forward
return originals.MultiheadAttention_forward(self, *args, **kwargs)

  File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\activation.py", line 1205, in forward
    attn_output, attn_output_weights = F.multi_head_attention_forward(

  File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\functional.py", line 5373, in multi_head_attention_forward
    attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal)

NameError: name 'scaled_dot_product_attention' is not defined`

actually, when I select the sdxl tensor in the UI, I get this error in the logs :

changing setting sd_model_checkpoint to sd_xl_base_1.0.safetensors [31e35c80fc]: NameError
(...)
File "C:\Users\nicolas\ai\sd.webui\webui\venv\lib\site-packages\torch\nn\functional.py", line 5373, in multi_head_attention_forward
    attn_output = scaled_dot_product_attention(q, k, v, attn_mask, dropout_p, is_causal)
NameError: name 'scaled_dot_product_attention' is not defined

and the tensor isn't shown as selected. I have to select it twice for it to stick

I relaunched the webui.bat and now I'm getting this error with sdxl base :
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

NicoBerlo · 2023-10-18T14:57:28Z

Ok, si I simply switched to the dev branch and it worked right away !

BBVamp · 2023-10-18T21:47:10Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

Run webui.bat - this should rebuild the virtual environment venv

When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

enter: venv\Scripts\activate.bat

the command line should now have (venv) shown at the beginning.

enter the following commands:

python.exe -m pip install --upgrade pip

python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

python -m pip uninstall -y nvidia-cudnn-cu11

venv\Scripts\deactivate.bat

webui.bat

Install the TensorRT extension using the Install from URL option

Once installed, go to the Extensions >> Installed tab and Apply and Restart

Worked like a charm.

hww44 · 2023-10-19T01:39:39Z

What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

Run webui.bat - this should rebuild the virtual environment venv

When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

enter: venv\Scripts\activate.bat

the command line should now have (venv) shown at the beginning.

enter the following commands:

python.exe -m pip install --upgrade pip

python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

python -m pip uninstall -y nvidia-cudnn-cu11

venv\Scripts\deactivate.bat

webui.bat

Install the TensorRT extension using the Install from URL option

Once installed, go to the Extensions >> Installed tab and Apply and Restart

Thanks for the detailed steps, much appreciated. It worked!!

I can't install tensorrt.It give me a error.

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tensorrt

My pip is the newest.I don't know what's happen.

Dacrikka · 2023-10-19T08:15:51Z

Hi, my error is "different"

Dacrikka · 2023-10-19T08:16:38Z

*** Error loading script: trt.py
Traceback (most recent call last):
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in
import ui_trt
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in
from exporter import export_onnx, export_trt
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in
from utilities import Engine
File "G:\Edo\Programmi_Installati\Stable\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 22, in
import onnx_graphsurgeon as gs
ModuleNotFoundError: No module named 'onnx_graphsurgeon'

What is "onnx_graphsurgeon"? I installe CUDA 11.8... on a 4060 16gb

pixelpainter · 2023-10-20T22:36:23Z

Is there a fix for the portable version, this won't work!

AugmentedRealityCat · 2023-10-20T23:25:56Z

Installing the latest nvidia drivers was the solution for me.

Stas-ai · 2023-10-22T20:44:48Z

What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

Run webui.bat - this should rebuild the virtual environment venv

When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

enter: venv\Scripts\activate.bat

the command line should now have (venv) shown at the beginning.

enter the following commands:

python.exe -m pip install --upgrade pip

python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

python -m pip uninstall -y nvidia-cudnn-cu11

venv\Scripts\deactivate.bat

webui.bat

Install the TensorRT extension using the Install from URL option

Once installed, go to the Extensions >> Installed tab and Apply and Restart

Thanks for the detailed steps, much appreciated. It worked!!

I can't install tensorrt.It give me a error.

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for tensorrt

My pip is the newest.I don't know what's happen.

Just run cmd in Administrator mode.

Zafflower · 2023-10-24T16:05:07Z

HI racerx2oo3,
Thanks so much for this answer. I had a different "not found" problem: Every time I launched webAI-user.bat after installing TensorRt, I got a popup error message that begin: "The procedure entry point ?destroyTensorDescriptorEx..." This message would popup just when the UI was launching and I would have to click multiple times to get rid of it. I followed your directions, and the popup went away. Thanks!
Zaffer

Ca0-10 · 2023-10-25T19:07:41Z

sinand99 · 2023-10-25T23:33:33Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

* Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

* Run webui.bat - this should rebuild the virtual environment venv

* When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

* enter:  venv\Scripts\activate.bat

* the command line should now have (venv) shown at the beginning.

* enter the following commands:

* * python.exe -m pip install --upgrade pip

* * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

* * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

* * python -m pip uninstall -y nvidia-cudnn-cu11

* * venv\Scripts\deactivate.bat

* webui.bat

* Install the TensorRT extension using the Install from URL option

* Once installed, go to the Extensions >> Installed tab and Apply and Restart

there is no venv folder in portable version. I followed all steps but it does not install. Gets stuck in "onnx_graphsurgeon" step. Nvidia should fix it, this is a mess.

racerx2oo3 · 2023-10-26T00:23:47Z

Sinan, Try this for the portable version. in file explorer open your sd.webui folder >> open the webui folder * In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists Open a command prompt and navigate to our base SD webui folder: For the portable version this would be: sd.webui * From the command line run: environment.bat * cd webui * enter the following commands: * * python.exe -m pip install --upgrade pip * * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir * * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir * * python -m pip uninstall -y nvidia-cudnn-cu11 * webui.bat * Install the TensorRT extension using the Install from URL option * Once installed, go to the Extensions >> Installed tab and Apply and Restart

…

On Wed, Oct 25, 2023 at 4:33 PM Sinan Dinç ***@***.***> wrote: What appears to have worked for others. From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case). * In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists * Delete the venv folder Open a command prompt and navigate to the base SD webui folder * Run webui.bat - this should rebuild the virtual environment venv * When the WebUI appears close it and close the command prompt Open a command prompt and navigate to the base SD webui folder * enter: venv\Scripts\activate.bat * the command line should now have (venv) shown at the beginning. * enter the following commands: * * python.exe -m pip install --upgrade pip * * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir * * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir * * python -m pip uninstall -y nvidia-cudnn-cu11 * * venv\Scripts\deactivate.bat * webui.bat * Install the TensorRT extension using the Install from URL option * Once installed, go to the Extensions >> Installed tab and Apply and Restart there is no venv folder in portable version. I followed all steps but it does not install. Gets stuck in "onnx_graphsurgeon" step. Nvidia should fix it, this is a mess. — Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANROKD3S7UNC2N5JKGHF2EDYBGOVRAVCNFSM6AAAAAA6EWWYWKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBQGE4TOMZYGQ> . You are receiving this because you commented.Message ID: ***@***.***>

contentis · 2023-10-30T08:32:55Z

Closing duplicates + Seems to be resolved.

HeavyTony2 · 2023-10-31T18:33:40Z

I'm facing the same issue. I deleted the VENV dir and followed the proposed procedure, but the output remains the same


HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
Exporting v1-5-pruned-emaonly to TensorRT
{'sample': [(1, 4, 64, 64), (2, 4, 64, 64), (8, 4, 96, 96)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
INFO:root:Exporting to ONNX...
Exporting to ONNX...
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:invalid unordered_map<K, T> key
invalid unordered_map<K, T> key
Traceback (most recent call last):
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 84, in export_onnx
    torch.onnx.export(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 506, in export
    _export(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 1548, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 1113, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 989, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 893, in _trace_and_get_graph_from_model
    trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 1268, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 127, in forward
    graph, out = torch._C._create_graph_by_tracing(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 118, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\modules\sd_unet.py", line 91, in UNetModel_forward
    return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
    h = module(h, emb, context)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 82, in forward
    x = layer(x, emb)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Hypernetwork-MonkeyPatch-Extension\patches\external_pr\sd_hijack_checkpoint.py", line 11, in ResBlock_forward
    return checkpoint(self._forward, x, emb)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\utils\checkpoint.py", line 249, in checkpoint
    return CheckpointFunction.apply(function, preserve, *args)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
RuntimeError: invalid unordered_map<K, T> key

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
    export_onnx(
  File "C:\Users\user\Documents\TestSD\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "C:\ProgramData\anaconda3\lib\_sitebuiltins.py", line 26, in __call__
    raise SystemExit(code)
SystemExit: None
INFO:httpx:HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 500 Internal Server Error"
HTTP Request: POST http://127.0.0.1:7860/api/predict "HTTP/1.1 500 Internal Server Error"
INFO:httpx:HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"
HTTP Request: POST http://127.0.0.1:7860/reset "HTTP/1.1 200 OK"

ajowens001 · 2023-12-30T00:25:17Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

* Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

* Run webui.bat - this should rebuild the virtual environment venv

* When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

* enter:  venv\Scripts\activate.bat

* the command line should now have (venv) shown at the beginning.

* enter the following commands:

* * python.exe -m pip install --upgrade pip

* * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

* * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

* * python -m pip uninstall -y nvidia-cudnn-cu11

* * venv\Scripts\deactivate.bat

* webui.bat

* Install the TensorRT extension using the Install from URL option

* Once installed, go to the Extensions >> Installed tab and Apply and Restart

Greetings,

I found this because I've tried multiple times to install this TRT extension while on a Linux OS (Ubuntu-based distro) with zero success. Each time I try to install via URL or from the extensions master list, the extension installation seems to hang, with an endless timer. I navigated to the ...webui/extensions/ folder and see that there is a 39.5 MB folder with the TRT files in there. Though, when I try to run ./webui.sh, the UI never loads - it just remains in the terminal, also looking as though it hangs.

My questions are: is this the same general issue as what is described in this issue thread? If so, can you provide the same guidance for Linux users? I don't believe that it's a problem with the venv; it really just seems to be an incomplete install. The reason I don't think this is a problem with the venv folder, is because I can delete the TRT extension folder and run the UI with no problem right after that deletion.

I am supposing that if I go into the .../venv/lib/python3.10/site-packages/ folder and ran the

    **python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
    **python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
    **python -m pip uninstall -y nvidia-cudnn-cu11

commands, it could be fixed, but I am not sure at all. I also don't know the Linux equivalent to the Windows commands you mentioned before, i.e. enter: venv\Scripts\activate.bat & venv\Scripts\deactivate.bat.

Can you please help?

andrewtvuong · 2024-01-06T23:01:16Z

What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

* Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
* Run webui.bat - this should rebuild the virtual environment venv

* When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
* enter:  venv\Scripts\activate.bat

* the command line should now have (venv) shown at the beginning.

* enter the following commands:

* * python.exe -m pip install --upgrade pip

* * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

* * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

* * python -m pip uninstall -y nvidia-cudnn-cu11

* * venv\Scripts\deactivate.bat

* webui.bat

* Install the TensorRT extension using the Install from URL option

* Once installed, go to the Extensions >> Installed tab and Apply and Restart
Greetings,

I found this because I've tried multiple times to install this TRT extension while on a Linux OS (Ubuntu-based distro) with zero success. Each time I try to install via URL or from the extensions master list, the extension installation seems to hang, with an endless timer. I navigated to the ...webui/extensions/ folder and see that there is a 39.5 MB folder with the TRT files in there. Though, when I try to run ./webui.sh, the UI never loads - it just remains in the terminal, also looking as though it hangs.

My questions are: is this the same general issue as what is described in this issue thread? If so, can you provide the same guidance for Linux users? I don't believe that it's a problem with the venv; it really just seems to be an incomplete install. The reason I don't think this is a problem with the venv folder, is because I can delete the TRT extension folder and run the UI with no problem right after that deletion.

I am supposing that if I go into the .../venv/lib/python3.10/site-packages/ folder and ran the
    **python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
    **python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
    **python -m pip uninstall -y nvidia-cudnn-cu11
commands, it could be fixed, but I am not sure at all. I also don't know the Linux equivalent to the Windows commands you mentioned before, i.e. enter: venv\Scripts\activate.bat & venv\Scripts\deactivate.bat.

Can you please help?

I'm also on Ubuntu, found the issue and created a fix #225

Basically if you're stuck:

Delete stable-diffusion-webui/venv dir
Delete existing (if existing) extensions/Stable-Diffusion-WebUI-TensorRT
Rerun webui.sh to rebuild venv
source venv/bin/activate in the stable-diffusion-webui dir MAKE ABSOLUTELY SURE THIS IS ENABLED

pip uninstall -y tensorrt
pip install -y nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4 --no-cache-dir
pip uninstall -y nvidia-cudnn-cu11

deactivate
Proceed with normal installation of Nvidia TensorRT

--- alternative, not recommend unless you know what you're doing

Then you can either apply my fix 7c93523 to https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/blob/main/install.py and run python install.py

or using the A1111 SD UI install my repo instead https://github.com/andrewtvuong/Stable-Diffusion-WebUI-TensorRT as a temporary work around. Once the main package is fixed, I would uninstall and install the main again.

Wontfallo · 2024-01-14T00:05:49Z

If Nvidia is going to advertise this as a main feature of using their GPU, then they really need to take care of this. I'm reading 4 months worth of threads here all with the same issues everyone else seems to be hitting.
AT LEAST fix the documentation. It's lazy at best. Don't they have enough resources to hire someone to fix and maintain this?

Normally this type of troubleshooting crap wouldn't bother me, but this is from the company that I spent a lot stupid money on for their graphics card. This isn't just some Joe Blow repo, this is advertised on their snobby website, and at all over CSE 2024 That's what makes this so frustrating to me.

Fix the doc, if you think it's good enough, you're an elitist snob and should be ashamed of yourself.

┻━┻ ︵＼( °□° )／︵ ┻━┻

"How to use
Click on the “Generate Default Engines” button." oh is that? Should I just click the button? You know, the button. Ya dummies. just click it. Done.

sinand99 · 2024-01-14T15:07:06Z

If Nvidia is going to advertise this as a main feature of using their GPU, then they really need to take care of this. I'm reading 4 months worth of threads here all with the same issues everyone else seems to be hitting. AT LEAST fix the documentation. It's lazy at best. Don't they have enough resources to hire someone to fix and maintain this?

Normally this type of troubleshooting crap wouldn't bother me, but this is from the company that I spent a lot stupid money on for their graphics card. This isn't just some Joe Blow repo, this is advertised on their snobby website, and at all over CSE 2024 That's what makes this so frustrating to me.

Fix the doc, if you think it's good enough, you're an elitist snob and should be ashamed of yourself.

┻━┻ ︵＼( °□° )／︵ ┻━┻

"How to use Click on the “Generate Default Engines” button." oh is that? Should I just click the button? You know, the button. Ya dummies. just click it. Done.

+1

FurkanGozukara · 2024-01-15T22:57:37Z

well for those who can't still make I made a full tutorial and also auto installers

Double Your Stable Diffusion Inference Speed with RTX Acceleration TensorRT: A Comprehensive Guide

fenyadim · 2024-01-19T17:59:48Z

What appears to have worked for others.
From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).
* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

* Delete the venv folder
Open a command prompt and navigate to the base SD webui folder
* Run webui.bat - this should rebuild the virtual environment venv

* When the WebUI appears close it and close the command prompt
Open a command prompt and navigate to the base SD webui folder
* enter:  venv\Scripts\activate.bat

* the command line should now have (venv) shown at the beginning.

* enter the following commands:

* * python.exe -m pip install --upgrade pip

* * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

* * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

* * python -m pip uninstall -y nvidia-cudnn-cu11

* * venv\Scripts\deactivate.bat

* webui.bat

* Install the TensorRT extension using the Install from URL option

* Once installed, go to the Extensions >> Installed tab and Apply and Restart
Greetings,
I found this because I've tried multiple times to install this TRT extension while on a Linux OS (Ubuntu-based distro) with zero success. Each time I try to install via URL or from the extensions master list, the extension installation seems to hang, with an endless timer. I navigated to the ...webui/extensions/ folder and see that there is a 39.5 MB folder with the TRT files in there. Though, when I try to run ./webui.sh, the UI never loads - it just remains in the terminal, also looking as though it hangs.
My questions are: is this the same general issue as what is described in this issue thread? If so, can you provide the same guidance for Linux users? I don't believe that it's a problem with the venv; it really just seems to be an incomplete install. The reason I don't think this is a problem with the venv folder, is because I can delete the TRT extension folder and run the UI with no problem right after that deletion.
I am supposing that if I go into the .../venv/lib/python3.10/site-packages/ folder and ran the
    **python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
    **python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
    **python -m pip uninstall -y nvidia-cudnn-cu11
commands, it could be fixed, but I am not sure at all. I also don't know the Linux equivalent to the Windows commands you mentioned before, i.e. enter: venv\Scripts\activate.bat & venv\Scripts\deactivate.bat.
Can you please help?
I'm also on Ubuntu, found the issue and created a fix #225

Basically if you're stuck:

Delete stable-diffusion-webui/venv dir

Delete existing (if existing) extensions/Stable-Diffusion-WebUI-TensorRT

Rerun webui.sh to rebuild venv

source venv/bin/activate in the stable-diffusion-webui dir MAKE ABSOLUTELY SURE THIS IS ENABLED
pip uninstall -y tensorrt
pip install -y nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir
pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4 --no-cache-dir
pip uninstall -y nvidia-cudnn-cu11
deactivate

Proceed with normal installation of Nvidia TensorRT

--- alternative, not recommend unless you know what you're doing

Then you can either apply my fix 7c93523 to https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/blob/main/install.py and run python install.py

or using the A1111 SD UI install my repo instead https://github.com/andrewtvuong/Stable-Diffusion-WebUI-TensorRT as a temporary work around. Once the main package is fixed, I would uninstall and install the main again.

Thank you!

ErixStrong · 2024-01-25T17:37:05Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

Run webui.bat - this should rebuild the virtual environment venv

When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

enter: venv\Scripts\activate.bat

the command line should now have (venv) shown at the beginning.

enter the following commands:

python.exe -m pip install --upgrade pip

python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

python -m pip uninstall -y nvidia-cudnn-cu11

venv\Scripts\deactivate.bat

webui.bat

Install the TensorRT extension using the Install from URL option

Once installed, go to the Extensions >> Installed tab and Apply and Restart

I tried to run this procedure several times, but it doesn't work fro me, I'm still getting this error:

*** Error running install.py for extension F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "F:\Stable-Diffusion\webui\venv\Scripts\python.exe" "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stderr: Traceback (most recent call last):
*** File "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in
*** from importlib_metadata import version
*** ModuleNotFoundError: No module named 'importlib_metadata'

CGGermany · 2024-01-28T00:13:38Z

I followed the instructions and it worked. 100%. In my case, the fifth point of the instructions (venv\Scripts\deactivate.bat) did not work and/or was not found. I then skipped it (omitted it). Thanks for the precise instructions. Now I have the TensorRT tab in the UI.

I previously set the arguments to default (deleted and kept backup). I have no idea if it has any meaning or use, but I also backed up the venv folder before deleting it.

reziozhou · 2024-02-07T03:38:29Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

Run webui.bat - this should rebuild the virtual environment venv

When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

enter: venv\Scripts\activate.bat

the command line should now have (venv) shown at the beginning.

enter the following commands:

python.exe -m pip install --upgrade pip

python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

python -m pip uninstall -y nvidia-cudnn-cu11

venv\Scripts\deactivate.bat

webui.bat

Install the TensorRT extension using the Install from URL option

Once installed, go to the Extensions >> Installed tab and Apply and Restart

I use the [Stable-Diffusion-WebUI-TensorRT] branch master but get errors caused by dll error,but changed to controlnet_v2 branch get fix it

Yomisana · 2024-02-28T10:24:36Z

For me msi rtx 2080 ti gaming x trio

issue

the procedure entry point 
?destroyTensorDescriptorEx@...
.....
...venv\Lib\Site-packages\nvidia\cudnn\bin\cudnn_adv_infer64_8.dll

how to fixed

close Stable Diffusion Webui
$> cd stable_diffusion_2080ti(This need change for your sd folder name)\venv\Scripts
$> cmd
$> activate.bat
$> python.exe -m pip install --upgrade pip
$> python -m pip uninstall -y nvidia-cudnn-cu11
open sd not gonna pop the error window again.🚀

Danny-707 · 2024-03-03T18:48:59Z

I tried to run this procedure several times, but it doesn't work fro me, I'm still getting this error:

*** Error running install.py for extension F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT. *** Command: "F:\Stable-Diffusion\webui\venv\Scripts\python.exe" "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py" *** Error code: 1 *** stderr: Traceback (most recent call last): *** File "F:\Stable-Diffusion\webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in *** from importlib_metadata import version *** ModuleNotFoundError: No module named 'importlib_metadata'

I have the same problem(
I've gone through the procedure several times, but with no success. Even on a freshly installed Automatic1111...

Danny-707 · 2024-03-03T18:56:48Z

This appears during installation:

*** Error running install.py for extension S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "S:\stable-diffusion-webui\venv\Scripts\python.exe" "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stderr: Traceback (most recent call last):
***  File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in <module>
***   from importlib_metadata import version
*** ModuleNotFoundError: No module named 'importlib_metadata'

This happens when starting the web UI:

venv "S:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.8.0
Commit hash: bef51aed032c0aaa5cfd80445bc4cf0d85b408b5
*** Error running install.py for extension S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "S:\stable-diffusion-webui\venv\Scripts\python.exe" "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stderr: Traceback (most recent call last):
***   File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 3, in <module>
***     from importlib_metadata import version
*** ModuleNotFoundError: No module named 'importlib_metadata'
Launching Web UI with arguments:
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
*** Error loading script: lora.py
    Traceback (most recent call last):
      File "S:\stable-diffusion-webui\modules\scripts.py", line 527, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "S:\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\lora.py", line 7, in <module>
        import onnx
    ModuleNotFoundError: No module named 'onnx'

---
*** Error loading script: trt.py
    Traceback (most recent call last):
      File "S:\stable-diffusion-webui\modules\scripts.py", line 527, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "S:\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "S:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 8, in <module>
        from polygraphy.logger import G_LOGGER
    ModuleNotFoundError: No module named 'polygraphy'

wangfeng35 · 2024-03-09T15:55:02Z

polygraphy

see here: #286

riperbot · 2024-03-18T22:17:22Z

My rude fix: #143 (comment)

peteer01 · 2024-04-11T01:03:03Z

What appears to have worked for others.

From your base SD webui folder: (E:\Stable diffusion\SD\webui\ in your case).

* In the extensions folder delete: stable-diffusion-webui-tensorrt folder if it exists

* Delete the venv folder

Open a command prompt and navigate to the base SD webui folder

* Run webui.bat - this should rebuild the virtual environment venv

* When the WebUI appears close it and close the command prompt

Open a command prompt and navigate to the base SD webui folder

* enter:  venv\Scripts\activate.bat

* the command line should now have (venv) shown at the beginning.

* enter the following commands:

* * python.exe -m pip install --upgrade pip

* * python -m pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir

* * python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir

* * python -m pip uninstall -y nvidia-cudnn-cu11

* * venv\Scripts\deactivate.bat

* webui.bat

* Install the TensorRT extension using the Install from URL option

* Once installed, go to the Extensions >> Installed tab and Apply and Restart

I stumbled upon this in 2024 and used the following commands in case the newer packages help today. Here are the commands I used:

python.exe -m pip install --upgrade pip
python -m pip install nvidia-cudnn-cu12==9.0.0.312 --no-cache-dir
python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.3.0.post12.dev1 --no-cache-dir
python -m pip uninstall -y nvidia-cudnn-cu12
venv\Scripts\deactivate.bat

If you use version 10 of nvidia-cudnn-cu12, you will get an error when trying to use TensorRT later, so this uses the latest version of 9.

You can also use the commands in the 2023 post above with the older packages. The older models worked, but I was getting the below errors after using the older packages, although TensorRT seemed to work:

[E] 1: [myelinCache.cpp::nvinfer1::builder::MyelinAutotunerCache::deserialize::62] Error Code 1: Myelin (Myelin error from unknown graph)
[E] 4: The timing cache will not be used!

I do receive "[E] 4: The timing cache will not be used!" regardless of what packages I use.

FurkanGozukara · 2024-10-01T02:33:32Z

@peteer01 how did you solve this error

    from utilities import Engine
  File "R:\Auto1111_V9\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 37, in <module>
    TRT_LOGGER = trt.Logger(trt.Logger.ERROR)
AttributeError: module 'tensorrt' has no attribute 'Logger'

kairin · 2024-12-07T02:08:26Z

I stumbled upon this in 2024 and used the following commands in case the newer packages help today. Here are the commands I used:

python.exe -m pip install --upgrade pip python -m pip install nvidia-cudnn-cu12==9.0.0.312 --no-cache-dir python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.3.0.post12.dev1 --no-cache-dir python -m pip uninstall -y nvidia-cudnn-cu12 venv\Scripts\deactivate.bat

If you use version 10 of nvidia-cudnn-cu12, you will get an error when trying to use TensorRT later, so this uses the latest version of 9.

stumbled upon what you stumbled upon in December of 2024.

the following steps helped me install Successfully installed tensorrt-9.3.0.post12.dev1 tensorrt_bindings-9.3.0.post12.dev1 tensorrt_libs-9.3.0.post12.dev1

have not confirmed if this solves my comfyUI issue with tensorrt...

.              .bashrc  .git-credentials  .landscape   .nv              .sudo_as_admin_successful  forge
..             .cache   .gitconfig        .lesshst     .profile         backupCustom_Nodes         snap
.bash_history  .config  .idlerc           .local       .pyenv           comfy
.bash_logout   .fonts   .keras            .motd_shown  .python_history  discord-msgs
Running (pyenv virtualenvs) this Lists all Python virtualenvs found in (/home/k/.pyenv/versions/*) in the next line
Running now.
  3.10.16/envs/ven1 (created from /home/k/.pyenv/versions/3.10.16)
  ven1 (created from /home/k/.pyenv/versions/3.10.16)
We are running (pyenv activate) to activate the listed environment (ven1) next.
Running now
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0
Sat Dec  7 10:03:03 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.72                 Driver Version: 566.14         CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4080 ...    On  |   00000000:0B:00.0  On |                  N/A |
|  0%   38C    P2             12W /  320W |     704MiB /  16376MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A        82      G   /Xwayland                                   N/A      |
|    0   N/A  N/A    453569      C   /python3.10                                 N/A      |
+-----------------------------------------------------------------------------------------+
VIRTUAL_ENV=/home/k/.pyenv/versions/3.10.16/envs/ven1
PYENV_VIRTUAL_ENV=/home/k/.pyenv/versions/3.10.16/envs/ven1
  3.10.16/envs/ven1 (created from /home/k/.pyenv/versions/3.10.16)
* ven1 (created from /home/k/.pyenv/versions/3.10.16)
.              .bashrc  .git-credentials  .landscape   .nv              .sudo_as_admin_successful  forge
..             .cache   .gitconfig        .lesshst     .profile         backupCustom_Nodes         snap
.bash_history  .config  .idlerc           .local       .pyenv           comfy
.bash_logout   .fonts   .keras            .motd_shown  .python_history  discord-msgs
.               CODEOWNERS       comfy_execution    extra_model_paths.yaml.example  node_helpers.py   styles
..              CONTRIBUTING.md  comfy_extras       fix_torch.py                    nodes.py          temp
.ci             LICENSE          comfyui.log        folder_paths.py                 notebooks         tests
.git            README.md        comfyui.prev.log   input                           output            tests-unit
.gitattributes  __pycache__      comfyui.prev2.log  latent_preview.py               pytest.ini        user
.github         api_server       cuda_malloc.py     main.py                         requirements.txt  utils
.gitignore      app              custom_nodes       models                          script_examples   web
.pylintrc       comfy            execution.py       new_updater.py                  server.py
(ven1) k@4080SUPER:~/comfy$ python -m pip install --upgrade pip
Requirement already satisfied: pip in /home/k/.pyenv/versions/3.10.16/envs/ven1/lib/python3.10/site-packages (24.3.1)
(ven1) k@4080SUPER:~/comfy$ python -m pip install nvidia-cudnn-cu12 --no-cache-dir
Requirement already satisfied: nvidia-cudnn-cu12 in /home/k/.pyenv/versions/3.10.16/envs/ven1/lib/python3.10/site-packages (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12 in /home/k/.pyenv/versions/3.10.16/envs/ven1/lib/python3.10/site-packages (from nvidia-cudnn-cu12) (12.4.5.8)
(ven1) k@4080SUPER:~/comfy$ python -m pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.3.0.post12.dev1 --no-cache-dir
Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com/
Collecting tensorrt==9.3.0.post12.dev1
  Downloading https://pypi.nvidia.com/tensorrt/tensorrt-9.3.0.post12.dev1.tar.gz (18 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tensorrt_libs==9.3.0.post12.dev1 (from tensorrt==9.3.0.post12.dev1)
  Downloading https://pypi.nvidia.com/tensorrt-libs/tensorrt_libs-9.3.0.post12.dev1-py2.py3-none-manylinux_2_17_x86_64.whl (1076.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 GB 12.5 MB/s eta 0:00:00
Collecting tensorrt_bindings==9.3.0.post12.dev1 (from tensorrt==9.3.0.post12.dev1)
  Downloading https://pypi.nvidia.com/tensorrt-bindings/tensorrt_bindings-9.3.0.post12.dev1-cp310-none-manylinux_2_17_x86_64.whl (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 12.5 MB/s eta 0:00:00
Requirement already satisfied: nvidia-cuda-runtime-cu12 in /home/k/.pyenv/versions/3.10.16/envs/ven1/lib/python3.10/site-packages (from tensorrt_libs==9.3.0.post12.dev1->tensorrt==9.3.0.post12.dev1) (12.4.127)
Requirement already satisfied: nvidia-cudnn-cu12 in /home/k/.pyenv/versions/3.10.16/envs/ven1/lib/python3.10/site-packages (from tensorrt_libs==9.3.0.post12.dev1->tensorrt==9.3.0.post12.dev1) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12 in /home/k/.pyenv/versions/3.10.16/envs/ven1/lib/python3.10/site-packages (from tensorrt_libs==9.3.0.post12.dev1->tensorrt==9.3.0.post12.dev1) (12.4.5.8)
Building wheels for collected packages: tensorrt
  Building wheel for tensorrt (pyproject.toml) ... done
  Created wheel for tensorrt: filename=tensorrt-9.3.0.post12.dev1-py2.py3-none-any.whl size=17623 sha256=a4c09483672557d445403c3815c7a73eff6bb71d004231e547473fcebd1eda5d
  Stored in directory: /tmp/pip-ephem-wheel-cache-rlbmxd6f/wheels/3e/f5/c9/323aa02815c27df8927e2a710d6d9b784684f5ae7077e44344
Successfully built tensorrt
Installing collected packages: tensorrt_bindings, tensorrt_libs, tensorrt
  Attempting uninstall: tensorrt
    Found existing installation: tensorrt 10.4.0
    Uninstalling tensorrt-10.4.0:
      Successfully uninstalled tensorrt-10.4.0
Successfully installed tensorrt-9.3.0.post12.dev1 tensorrt_bindings-9.3.0.post12.dev1 tensorrt_libs-9.3.0.post12.dev1
(ven1) k@4080SUPER:~/comfy$

anwoflow mentioned this issue Oct 18, 2023

Error during inference - RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! #18

Closed

contentis added bug Something isn't working duplicate This issue or pull request already exists labels Oct 18, 2023

This was referenced Oct 18, 2023

Please write a proper howto/readme, which doesnt leave out needed information #33

Closed

After install when i run webui.bat i got a lot of errors and dont work! #34

Closed

ModuleNotFoundError: No module named 'tensorrt_bindings' #41

Closed

GumGum10 mentioned this issue Oct 18, 2023

Unable to generate engine on dev branch #38

Closed

IndigoDosSantos mentioned this issue Oct 19, 2023

python.exe entry point not found #78

Closed

aquawaves mentioned this issue Oct 19, 2023

pls help to resolve this trouble #83

Open

thomasgaragan mentioned this issue Oct 25, 2023

Error loading script: trt.py #127

Closed

contentis closed this as completed Oct 30, 2023

This was referenced Oct 30, 2023

The procedure entry point could not be located in the DLL Library #143

Closed

AttributeError: module 'tensorrt' has no attribute 'IProgressMonitor' #144

Open

contentis mentioned this issue Jan 2, 2024

lora_v2 branch fails to start on A1111 V1.7.0 #207

Open

T9es mentioned this issue Jan 6, 2024

Error running install.py / SD version error #220

Closed

andrewtvuong mentioned this issue Jan 6, 2024

[Fix] Install.py silently blocked by blocking call of nvidia-cudnn-cu11 uninstall due to interactive python shell opening #225

Closed

This was referenced Jan 7, 2024

AttributeError: 'list' object has no attribute 'get' #227

Closed

Installation fails with permission issue on google\\~-pb\\_message.pyd #232

Open

Install failed part-way and now thinks it's installed, but doesn't work and can't uninstall #228

Closed

iKurama mentioned this issue Jan 9, 2024

PermissionError: [Errno 13] - model.json denied #234

Closed

ItsSkynet mentioned this issue Jan 10, 2024

Fails with RuntimeError: The expanded size of the tensor (1) must match the existing size (2) at non-singleton dimension 0. Target sizes: [1, 4, 64, 64]. Tensor sizes: [2, 4, 64, 64] #235

Open

Kertnik mentioned this issue Jan 13, 2024

TypeError: list indices must be integers or slices, not NoneType #242

Open

fireattack mentioned this issue Jan 25, 2024

这个怎么解决？ #247

Open

banjaminicc mentioned this issue Feb 15, 2024

Getting strange error popups after installation via extensions tab #274

Open

YS-YYDS mentioned this issue Jul 6, 2024

ONNX Export Error: Tensors on Different Devices (CPU and CUDA) #339

Open

kairin mentioned this issue Dec 7, 2024

Cannot import /home/k/comfy/custom_nodes/ComfyUI-Upscaler-Tensorrt module for custom nodes: No module named 'tensorrt_bindings' yuvraj108c/ComfyUI-Upscaler-Tensorrt#33

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error loading script: trt.py, ModuleNotFoundError: No module named 'tensorrt_bindings' #27

Error loading script: trt.py, ModuleNotFoundError: No module named 'tensorrt_bindings' #27

rlenci99 commented Oct 18, 2023

racerx2oo3 commented Oct 18, 2023

rlenci99 commented Oct 18, 2023

NicoBerlo commented Oct 18, 2023 •

edited

Loading

NicoBerlo commented Oct 18, 2023

BBVamp commented Oct 18, 2023

hww44 commented Oct 19, 2023 •

edited

Loading

Dacrikka commented Oct 19, 2023

Dacrikka commented Oct 19, 2023

pixelpainter commented Oct 20, 2023

AugmentedRealityCat commented Oct 20, 2023

Stas-ai commented Oct 22, 2023 •

edited

Loading

Zafflower commented Oct 24, 2023

Ca0-10 commented Oct 25, 2023

sinand99 commented Oct 25, 2023

racerx2oo3 commented Oct 26, 2023 via email

contentis commented Oct 30, 2023

HeavyTony2 commented Oct 31, 2023

ajowens001 commented Dec 30, 2023 •

edited

Loading

andrewtvuong commented Jan 6, 2024 •

edited

Loading

Wontfallo commented Jan 14, 2024 •

edited

Loading

sinand99 commented Jan 14, 2024

FurkanGozukara commented Jan 15, 2024

fenyadim commented Jan 19, 2024

ErixStrong commented Jan 25, 2024

CGGermany commented Jan 28, 2024

reziozhou commented Feb 7, 2024

Yomisana commented Feb 28, 2024

Danny-707 commented Mar 3, 2024

Danny-707 commented Mar 3, 2024

wangfeng35 commented Mar 9, 2024

riperbot commented Mar 18, 2024 •

edited

Loading

peteer01 commented Apr 11, 2024 •

edited

Loading

FurkanGozukara commented Oct 1, 2024

kairin commented Dec 7, 2024

Error loading script: trt.py, ModuleNotFoundError: No module named 'tensorrt_bindings' #27

Error loading script: trt.py, ModuleNotFoundError: No module named 'tensorrt_bindings' #27

Comments

rlenci99 commented Oct 18, 2023

racerx2oo3 commented Oct 18, 2023

rlenci99 commented Oct 18, 2023

NicoBerlo commented Oct 18, 2023 • edited Loading

NicoBerlo commented Oct 18, 2023

BBVamp commented Oct 18, 2023

hww44 commented Oct 19, 2023 • edited Loading

Dacrikka commented Oct 19, 2023

Dacrikka commented Oct 19, 2023

pixelpainter commented Oct 20, 2023

AugmentedRealityCat commented Oct 20, 2023

Stas-ai commented Oct 22, 2023 • edited Loading

Zafflower commented Oct 24, 2023

Ca0-10 commented Oct 25, 2023

sinand99 commented Oct 25, 2023

racerx2oo3 commented Oct 26, 2023 via email

contentis commented Oct 30, 2023

HeavyTony2 commented Oct 31, 2023

ajowens001 commented Dec 30, 2023 • edited Loading

andrewtvuong commented Jan 6, 2024 • edited Loading

Wontfallo commented Jan 14, 2024 • edited Loading

sinand99 commented Jan 14, 2024

FurkanGozukara commented Jan 15, 2024

fenyadim commented Jan 19, 2024

ErixStrong commented Jan 25, 2024

CGGermany commented Jan 28, 2024

reziozhou commented Feb 7, 2024

Yomisana commented Feb 28, 2024

issue

how to fixed

Danny-707 commented Mar 3, 2024

Danny-707 commented Mar 3, 2024

wangfeng35 commented Mar 9, 2024

riperbot commented Mar 18, 2024 • edited Loading

peteer01 commented Apr 11, 2024 • edited Loading

FurkanGozukara commented Oct 1, 2024

kairin commented Dec 7, 2024

NicoBerlo commented Oct 18, 2023 •

edited

Loading

hww44 commented Oct 19, 2023 •

edited

Loading

Stas-ai commented Oct 22, 2023 •

edited

Loading

ajowens001 commented Dec 30, 2023 •

edited

Loading

andrewtvuong commented Jan 6, 2024 •

edited

Loading

Wontfallo commented Jan 14, 2024 •

edited

Loading

riperbot commented Mar 18, 2024 •

edited

Loading

peteer01 commented Apr 11, 2024 •

edited

Loading