I have implemented stitchdiffusion based on diffusers to generate 360-degree panoramas through text prompts. The detailed steps of installation and model inference are in the following.

[Attention] This colab is not working due to the updated google colab again, I don't plan to fix it.

[Next Step] To achieve 2048x1024 (2K) 360-degree panorama based on SDXL+StitchDiffusion in Feb 2025.

1. Create Environment

Recommend to use Conda here.

conda create -n stitchdiffusion python=3.9
conda activate stitchdiffusion
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=11.8 -c pytorch -c nvidia

2. Packages Installation

Step 1.

git clone https://github.com/lshus/stitchdiffusion-colab.git

Step 2.

pip install -r requirements.txt

3. Download LoRA file

Step 1. Download link of LoRA from StitchDiffusion

Step 2. (Since I implemented it based on diffusers, some key names in the LoRA require to be changed for consistency. Before you run lora_modified.py, remember to modify "YOUR_PATH" in this file. Finally, you can get a file called "lora_modified.safetensors", which will be used later.)

python lora_modified.py

4. 360 Panorama Generation

Step 1. Remeber to change "YOUR_PATH" in this stitchdiffusion_360_gen.py.

Step 2. Specify the input prompt. (360-degree panoramic image is the trigger word, if you want to generate a 360-degree panorama, the input prompt should include it.)

Step 3. Run the file.

python stitchdiffusion_360_gen.py

Acknowledgement

Many thanks to kohya-trainer, StitchDiffusion and diffusers for their open-source code.