Adding support for GIMP 3.0 (#159)

* Set config and model paths dynamically from environment vars Signed-off-by: Will French <[email protected]> * Centralizing ENV VARs into tools_utils. * Fixing default path. * fixing test for python executable logic * Adding threading to SR to avoid not responding messages. * removing debug statement * Improvements to ease packaging work on Linux (#156) * Allow model_setup to be run from any directory * Store plugin scripts with executable permissions * Remove typing as it ships with stdlib since python 3.5 * Install plugin files with executable permissions preserved * Revert data_files addition to avoid complexity * Install plugins from source repo instead of venv * Revert permissions changes to test failure inside snap * Initial update of SR to GIMP 3.0. * Updating Save Image to work with GIMP 3.0 * Removing unneeded comments. * Initial update of Segmentation to GIMP 3.0 * Removing uneeded files. * Updating install path * Moving to latest OpenVINO - 2024.6.0 * removing unneeded log.info calls. * Updating proc id for GIMP 3.0 * Updating plugin version number * Initial update of SD to GIMP 3.0 * Updating option caching in SD. * removed files that are no longer used. * Implemented SDOptionCache to address issues with options being saved run-to-run. * Removed debug prints, unneeded syspath extention * remove commented syspath line * Adding version support for plugins. * updated install flow. removed need to call complete_install separately. * Hiding console for SR and SS. SD is still WIP * removed .json config, as it should be created from scratch everytime. * restored alignment of OV logo. * Updated NPU Arch logic to support next gen NPUs * Updated NPU Arch logic to support next gen NPUs * update ARCH_NEXT and tweak logic * Fixed NPU Arch logic to work correctly. * Add check for dNPU to avoid NPU turbo enabling exception for those devices * workaround for 3700 series being reported incorrectly in OV2024.5+ * restore running complete_install post setup * update tools_utils import * subprocess call is diff for Linux vs Windows * Removing import of install_utils. It relies on importing openvino, which was causing issues in Linux for some reason. For now, we are have the get_plugin_version code duplicated here. Need to find a way to remove it in the future. * Longer timeout needed for some systems. * Updating title bar to show just the plugin version. * updating process name to work with Linux * Adding linux support for BIOS discovery * Really fixing issue #158 this time. * Updating linux build docs * fixing issue with ENV var installation and commandline model installs. * Updating download links for GIMP3 RC2 * Updating figure for GIMP3 RC2 * Updating setup complete figure for GIMP3 RC2 * Updates for Docs and Figures for GIMP 3.0 --------- Signed-off-by: Garth Long <[email protected]> Signed-off-by: Will French <[email protected]> Co-authored-by: Will French <[email protected]> Co-authored-by: Ryan Metcalfe <[email protected]>
intel · Jan 11, 2025 · 6c5284c · 6c5284c
1 parent e7395e3
commit 6c5284c
Show file tree

Hide file tree

Showing 50 changed files with 1,045 additions and 1,313 deletions.
diff --git a/Docs/GIMP_StableDiffusion_Usage.md b/Docs/GIMP_StableDiffusion_Usage.md
@@ -0,0 +1,65 @@
+
+
+# Using Stable Diffusion in GIMP
+
+After installing GIMP and the GIMP AI plugins with OpenVINO™, you can now start GIMP application from Start Menu on Windows, or from the Application Launcher in Ubuntu. 
+
+>Notes: This section runs `SD-1.5 Square (512x512)` as an example. 
+
+## Download & Install Stable Diffusion Models
+- Once GIMP application is started, Click "__Layer__" \> "__OpenVINO-AI-Plugins__" \> "__Stable diffusion__".
+- Once Stable Diffusion Plugin UI opens up, click on Model Button on the top left corner as show in the below picture:
+
+    ![](figs/stable-diffusion_model1.png)
+- This now opens the Stable Diffusion Model Management window from where you can install the required models by clicking on "Install" button next to the model name. 
+
+    ![](figs/stable-diffusion_model2.png)  
+
+ - Download Stable Diffusion 1.5 Square as an instance. 
+
+    ![](figs/stable-diffusion_model3.png)
+
+ - You can select multiple models at once, and the Model Manager will queue them for installation.
+
+   ![](figs/stable-diffusion_model3_1.png)
+
+ - Once installation process is completed, you will see the "Install" button is updated to "Installed" and you will find the models in the drop-down list
+
+    ![](figs/stable-diffusion_model4.png)
+
+    >**Notes:**
+    > - The downloaded models include INT8, INT8A16, and FP16 precision
+    > - Weights is saved at `C:\Users\<user_name>\openvino-ai-plugins-gimp\weights`.
+
+
+
+## Execute Stable-Diffusion – Stable Diffusion 1.5 INT8A16
+
+Please follow below steps to execute Stable-Diffusion - Stable Diffusion 1.5 INT8A16. For other features, please refer to [OpenVINO™ Image Generator Plugin with Stable Diffusion](https://github.com/intel/openvino-ai-plugins-gimp/tree/main?tab=readme-ov-file#openvino-image-generator-plugin-with-stable-diffusion) section.
+
+- Following previous section to launch GIMP
+
+    ![](figs/gimp_launch.png)
+
+- Click "__Layer__" \> "__OpenVINO-AI-Plugins__" \> "__Stable diffusion__".
+
+    ![](figs/gimp_execute_SD.png)
+
+- Change the selected fields to set "Stable Diffusion" configuration and choose the desired "Power Mode" you want, then click "Load Models" to the load models into the target devices based on your power mode selection. 
+
+    ![](figs/gimp_load_model.png)
+
+    > **Notes:** It takes time in this step. 
+
+- Finally, you can optionally enter any text or changes the parameters in the selected field, then click "Generate" to generate image.
+
+    ![](figs/gimp_sd_ui.png)
+
+    >**Notes:**
+    > - Only Intel® devices are supported (CPU, NPU, integraged GPU, Intel® discrete GPUs)
+    > - Power Mode is now enabled- Users can select between the following options depending on their use case:
+    >   - Best Performance
+    >   - Best Power Efficiency
+    >   - Balanced
+    > - If you wish to generate more images in single run, please modify the Number of Images section.
+
diff --git a/Docs/figs/gimp_execute_SD.png b/Docs/figs/gimp_execute_SD.png
diff --git a/Docs/figs/gimp_installation_finish.png b/Docs/figs/gimp_installation_finish.png
diff --git a/Docs/figs/gimp_installation_install.png b/Docs/figs/gimp_installation_install.png
diff --git a/Docs/figs/gimp_launch.png b/Docs/figs/gimp_launch.png
diff --git a/Docs/figs/gimp_load_model.png b/Docs/figs/gimp_load_model.png
diff --git a/Docs/figs/gimp_sd_ui.png b/Docs/figs/gimp_sd_ui.png
diff --git a/Docs/figs/stable-diffusion_3.png b/Docs/figs/stable-diffusion_3.png
diff --git a/Docs/figs/stable-diffusion_model1.png b/Docs/figs/stable-diffusion_model1.png
diff --git a/Docs/figs/stable-diffusion_model2.png b/Docs/figs/stable-diffusion_model2.png
diff --git a/Docs/figs/stable-diffusion_model3.png b/Docs/figs/stable-diffusion_model3.png
diff --git a/Docs/figs/stable-diffusion_model3_1.png b/Docs/figs/stable-diffusion_model3_1.png
diff --git a/Docs/figs/stable-diffusion_model4.png b/Docs/figs/stable-diffusion_model4.png
diff --git a/Docs/linux_install_guide.md b/Docs/linux_install_guide.md
@@ -14,7 +14,7 @@
     ```sh
     git clone https://gitlab.gnome.org/GNOME/babl
     cd babl
-    git checkout tags/BABL_0_1_98
+    git checkout tags/BABL_0_1_110
     meson _build
     ninja -C _build
     sudo ninja -C _build install
@@ -24,7 +24,7 @@
     ```sh
     git clone https://gitlab.gnome.org/GNOME/gegl
     cd gegl
-    git checkout tags/GEGL_0_4_46
+    git checkout tags/GEGL_0_4_52
     meson _build
     ninja -C _build
     sudo ninja -C _build install
@@ -35,7 +35,9 @@
     ```sh
     git clone https://gitlab.gnome.org/GNOME/gimp  
     cd gimp
-    git checkout tags/GIMP_2_99_16
+    git checkout tags/GIMP_3_0_0_RC2
+    git submodule update --init 
+    export GI_TYPELIB_PATH=/usr/lib/x86_64-linux-gnu/girepository-1.0:/usr/local/lib/x86_64-linux-gnu/girepository-1.0
     meson _build
     ninja -C _build
     sudo ninja -C _build install
@@ -58,7 +60,7 @@
  Start GIMP, ensuring to setup the environment variables correctly,  and you should see 'OpenVINO-AI-Plugins' show up in 'Layer' menu
    ```sh
    export GI_TYPELIB_PATH=/usr/lib/x86_64-linux-gnu/girepository-1.0:/usr/local/lib/x86_64-linux-gnu/girepository-1.0
-   gimp-2.99
+   gimp
    ```
 
 
diff --git a/Docs/stable-diffusion-v3.md b/Docs/stable-diffusion-v3.md
@@ -31,7 +31,7 @@ Verify that the copied files and directory structure looks as follows:
 ![image](https://github.com/user-attachments/assets/039073d1-e593-4365-92c9-3555ea023670)
 ### Running with GIMP
 After completing model installation steps, SD3 will now be available in the Stable Diffusion UI. Note that SD3 can generate valid images in as few as 4 iterations. Also, the guidance scale needed is normally much lower. See the screenshot below:
-![image](https://github.com/user-attachments/assets/6daf3201-b873-4198-a752-19594f352c50)
+![](figs/stable-diffusion_3.png)
 
 
 

diff --git a/Docs/user_guide_for_windows_users.md b/Docs/user_guide_for_windows_users.md
@@ -2,24 +2,26 @@
 
 # User guide for Windows Users
 
-## Install GIMP AI plugins with OpenVINO™ Backend on 
+## Install GIMP AI plugins with OpenVINO™ 
 
->__Notes:__ To get NPU support, please following below configurations. If you are not seeking NPU version, you also can run this pulgin on any Intel CPU and GPU. Please note only Intel's ARC GPU is supported incase you have an external GPU. 
+>__Notes:__ 
+>This set of plugins only supports Intel® devices. Non-Intel GPUs and NPUs are not supported. 
+>NPU is supported on all Intel® Core™ Ultra series processors. 
 
-### Pre-requisite for execution on Intel NPU
+### Pre-requisite for execution on Intel® NPU
 
 - Hardware:
-  - Intel Core Ultra platform
+  - Intel® Core™ Ultra platform
   - 16GB system memory as minimum requirement
   - internet connection is required for installation
 - Driver:
-  - Intel NPU driver: Use the most recent driver you have available. 
+  - Intel® NPU driver: Use the most recent driver you have available. 
 - Software and Package:
   - git
   - python 3.9-3.12
     - Note: This document will use python 3.9.13 as an example.
   - VC runtime
-  - [GIMP 2.99.16](https://download.gimp.org/gimp/v2.99/windows/gimp-2.99.16-setup.exe)
+  - [GIMP 3.0 RC2](https://download.gimp.org/gimp/v3.0/windows/gimp-3.0.0-RC2-setup.exe)
   - [GIMP AI plugins with OpenVINO™ Backend](https://github.com/intel/openvino-ai-plugins-gimp) from Github.
 
 
@@ -70,9 +72,9 @@ Please download the latest Visual C++ Redistributable package from MSFT [site](h
 ![](figs/VC_runtime_close.png) 
 
 
-### Install GIMP 2.99.16
+### Install GIMP 3.0 RC2
 
-Please download [gimp-2.99.16-setup.exe](https://download.gimp.org/gimp/v2.99/windows/gimp-2.99.16-setup.exe) and follow below steps to install GIMP.
+Please download [GIMP 3.0 RC2](https://download.gimp.org/gimp/v3.0/windows/gimp-3.0.0-RC2-setup.exe) and follow below steps to install GIMP.
 
 - Click "Install for all users (recommended)"
 
@@ -97,8 +99,7 @@ Please download [gimp-2.99.16-setup.exe](https://download.gimp.org/gimp/v2.99/wi
 
 
 
-### Install GIMP AI plugins 
-
+### Install GIMP AI plugins
  - Open command prompt and download the latest release from Github by git with below instruction.
 
     ```sh
@@ -118,67 +119,4 @@ Please download [gimp-2.99.16-setup.exe](https://download.gimp.org/gimp/v2.99/wi
 
     This step will take time for downloading necessary packages.
 
-- After creating python environment, gimpenv3, and downloaded necessary packages, you can now start GIMP application from Start Menu. 
-
-# Execute Stable-Diffusion in GIMP
-
->Notes: This section runs `SD-1.5 Square (512x512)` as an example. 
-
-With previous section, the GIMP AI plugins with OpenVINO™ Backend is installed and you can execute stable diffusion in GIMP. This session will guide you to execute Stable-Diffusion in GIMP.
-
-## Execute GIMP
-
-You can now simply open Gimp from the start menu as you would any normal Windows application.
-
-## Download & Install Stable Diffusion Models
-- Once GIMP application is started, Click "__Layer__" \> "__OpenVINO-AI-Plugins__" \> "__Stable diffusion__".
-- Once Stable Diffusion Plugin UI opens up, click on Model Button on the top left corner as show in the below picture:
-
-    ![](figs/stable-diffusion_model1.png)
-- This now opens the Stable Diffusion Model Management window from where you can install the required models by clicking on "Install" button next to the model name. 
-
-    ![](figs/stable-diffusion_model2.png)  
-
- - Download Stable Diffusion 1.5 Square as an instance. 
-
-    ![](figs/stable-diffusion_model3.png)
-
- - Once installation process is completed, you will see the "Install" button is updated to "Installed" and you will find the models in the drop-down list
-
-    ![](figs/stable-diffusion_model4.png)
-
-    >**Notes:**
-    > - The downloaded models include INT8, INT8A16, and FP16 precision
-    > - Weights is saved at `C:\Users\<user_name>\openvino-ai-plugins-gimp\weights`.
-
-
-
-## Execute Stable-Diffusion – Stable Diffusion 1.5 INT8A16
-
-Please follow below steps to execute Stable-Diffusion - Stable Diffusion 1.5 INT8A16. For other features, please refer to [OpenVINO™ Image Generator Plugin with Stable Diffusion](https://github.com/intel/openvino-ai-plugins-gimp/tree/main?tab=readme-ov-file#openvino-image-generator-plugin-with-stable-diffusion) section.
-
-- Following previous section to launch GIMP
-
-    ![](figs/gimp_launch.png)
-
-- Click "__Layer__" \> "__OpenVINO-AI-Plugins__" \> "__Stable diffusion__".
-
-    ![](figs/gimp_execute_SD.png)
-
-- Change the selected fields to set "Stable Diffusion" configuration and choose the desired "Power Mode" you want, then click "Load Models" to the load models into the target devices based on your power mode selection. 
-
-    ![](figs/gimp_load_model.png)
-
-    > **Notes:** It takes time in this step. 
-
-- Finally, you can optionally enter any text or changes the parameters in the selected field, then click "Generate" to generate image.
-
-    ![](figs/gimp_sd_ui.png)
-
-    >**Notes:**
-    > - Power Mode is now enabled- Users can select between the following options depending on their use case for more detail please check next session:
-    >   - Best Performance (Only Intel's ARC GPU supported incase there is an external GPU)
-    >   - Best Power Efficiency
-    >   - Balanced
-    > - If you wish to generate more images in single run, please modify the Number of Images section.
-
+- After creating python environment, gimpenv3, and downloaded necessary packages, you can now launch GIMP application. See the [Usage Guide](GIMP_StableDiffusion_Usage.md) for running Stable Diffusion example.
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,3 +1,4 @@
 include gimpopenvino/plugins/openvino_utils/images/plugin_logo.png
 include gimpopenvino/plugins/openvino_utils/images/error_icon.png
+include gimpopenvino/plugins/openvino_utils/tools/gimp_openvino_config.json
 
diff --git a/README.md b/README.md
@@ -3,14 +3,20 @@
 <br>Dedicated for GIMP 3, Python 3 and OpenVINO™.<br> :star: :star: :star: :star: are welcome.<br>
 
 ## Current list of plugins:
-[1] Super-Resolution <br>
-[2] Semantic-Segmentation <br>
-[3] Stable-Diffusion (Supports - SD 1.4, SD 1.5 (landscape, portrait, and Latent Consistency Model), SD 1.5 Inpainting, SD 1.5 Controlnet-OpenPose, SD 1.5 Controlnet-CannyEdge, SD 1.5 Controlnet-Scribble) <br>
+1. Super-Resolution 
+2. Semantic-Segmentation 
+3. Stable-Diffusion  
+    * SD 1.5 (landscape, portrait, and Latent Consistency Model)
+    * SD 1.5 Inpainting 
+    * SD 1.5 Controlnet-OpenPose 
+    * SD 1.5 Controlnet-CannyEdge 
+    * SD 1.5 Controlnet-Scribble 
+    * SD 3.0 w/ Flash SD3 LCM  
 
 # Objectives
-[1] Provides a set of OpenVINO™ based plugins that add AI features to GIMP. <br>
-[2] Serve as a reference code for how to make use of OpenVino in GIMP application for inferencing on Intel's CPU & GPU. <br>
-[3] Add AI to routine image editing workflows. <br>
+1. Provides a set of OpenVINO™ based plugins that add AI features to GIMP. 
+2. Serve as a reference code for how to make use of OpenVino in GIMP application for inferencing on Intel® Devices. 
+3. Add AI to routine image editing workflows. 
 
 # Contribution
 Welcome people interested in contribution!
@@ -24,7 +30,7 @@ Please raise a PR for any new features, modifications, or bug fixes.
 For detailed steps and tips please refer [User guide for Windows](./Docs/user_guide_for_windows_users.md).
 
 ### Linux
-This plugin is tested on Ubuntu 22.04. Building GIMP from source is recommended.<br>
+This plugin is tested on Ubuntu 24.04. Building GIMP from source is recommended. Flatpak is not supported.<br>
 For detailed steps and tips please refer to [Linux Installation Guide](./Docs/linux_install_guide.md).
 
 # OpenVINO™ Image Generator Plugin with Stable Diffusion
@@ -79,7 +85,8 @@ For detailed steps and tips please refer to [Linux Installation Guide](./Docs/li
 5. Follow steps 4,5 from section A.
 6. Click on “Generate”. Wait for the total inference steps to get completed.
 
-
+#### G. Stable-Diffusion-3.0 w/ Flash SD3
+Please see [Stable Diffusion 3 User Guilde](./Docs/stable-diffusion-v3.md) for details
 _With Advanced Setting and Power Mode_
 
 ![](gifs/stable-diffusion2.png)

diff --git a/gimpopenvino/complete_install.py b/gimpopenvino/complete_install.py