Jetson Nano support

[zeittresor]

Hi there,
is there already any working "package" (fork) to support the (old and/or the new Orin Nano) nvidia jetson ?
i can only find it for A1111 webui but not for directly for forge webui, i just ask before i try to dev it for me by myself.. -> https://github.com/dusty-nv/jetson-containers/tree/master/packages/diffusion/stable-diffusion-webui#user-content-containers

[toyssamurai]

I know this is not a real world performance metric, but NVIDIA itself states that the Orin Nano can deliver up to 67 TOPS only. I don't have a number for lower tier RTX GPUs, but NVIDIA has said that a 4090 offers more than 1,300 TOPS. I doubt the Nano has the processing power to handle anything above SDXL. And, I haven't begun to mention the VRAM situation.

[zeittresor]

Hi, the reason i want to use the jetson nano is the energy efficience of the device (not the performance), it just need 7 to max. 25 W to work. i just want to combine it with a 3D holographic display fan using forge with the layer diffusion extension to generate random images (with transparent alpha channel) using "generate forever" feature, for a energy efficent indipendent shop window display.

[BlueBackbite]

In my experience the Jetson platform runs models fairly well, particularly the AGX Orin at 64GB of memory. The Orin diffuses as fast as my 3080Ti. Can't speak to how well exactly the Orin Nano would perform, particularly with larger models.

I started work on a docker container port for jetpack 6.1, but CUDA 12.6 is incompatible with xformers v0.0.27 it would seem.

[BlueBackbite]

I've posted my ugly dockerfile that builds sd-forge on jetpack 6.1 here:
https://github.com/BlueBackbite/Jetpack-sd-webui-forge/blob/main/Dockerfile

Commented sections left in as an example for anyone running into dependency conflicts later on.

The only issue I'm running into is a strange memory leak (?)

Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]
Version: f2.0.1v1.10.1-previous-635-gf5330788
Commit hash: f53307881bfd824dbdce6ac0d4bba04d9a74ab36
Installing clip
Cloning assets into /opt/sd-webui-forge/repositories/stable-diffusion-webui-assets...
Cloning into '/opt/sd-webui-forge/repositories/stable-diffusion-webui-assets'...
Cloning huggingface_guess into /opt/sd-webui-forge/repositories/huggingface_guess...
Cloning into '/opt/sd-webui-forge/repositories/huggingface_guess'...
Cloning google_blockly into /opt/sd-webui-forge/repositories/google_blockly_prototypes...
Cloning into '/opt/sd-webui-forge/repositories/google_blockly_prototypes'...
Cloning BLIP into /opt/sd-webui-forge/repositories/BLIP...
Cloning into '/opt/sd-webui-forge/repositories/BLIP'...
Installing requirement for task-scheduler
Installing requirements for webui-qrcode-generator
Installing requirements for CivitAI Browser
Installing requirements for CivitAI Browser
Installing requirements for CivitAI Browser
Installing requirements for CivitAI Browser
Installing requirements for CivitAI Browser
Installing None

at about this point memory starts to balloon until OOM.
removing extensions seems to still cause a memory leak condition without any information output. I seem to be having an issue with --verbose.