Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no kernel image is available for execution on the device at line 167 in file D:\ai\tool\bitsandbytes\csrc\ops.cu #47

Closed
RoyHammerlin opened this issue Nov 8, 2022 · 10 comments

Comments

@RoyHammerlin
Copy link

Getting this error from bitsandbytes when running dreambooth in auto1111. I have seen the fix listed here but it's not clear how to implement it in Windows.

GPU is a 1080ti with 11GB VRAM

CUDA SETUP: Loading binary C:\SD\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll...
 Scheduler Loaded
 Allocated: 0.3GB
 Reserved: 0.3GB

  Total target lifetime optimization steps = 1000
 CPU: False Adam: True, Prec: no, Prior: True, Grad: True, TextTr: True
 Allocated: 4.0GB
 Reserved: 4.1GB

Steps:   0%|                                                                                  | 0/1000 [00:00<?, ?it/s]Error no kernel image is available for execution on the device at line 167 in file D:\ai\tool\bitsandbytes\csrc\ops.cu
Press any key to continue . . .
@RoyHammerlin
Copy link
Author

RoyHammerlin commented Nov 8, 2022

I tried the steps mentioned in this post. It did take a little time installing some components for dreambooth, but I am still getting the same error, with a little more detail after the fault:

CUDA SETUP: Loading binary C:\SD\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda116.dll...
 Scheduler Loaded
 Allocated: 0.3GB
 Reserved: 0.3GB

  Total target lifetime optimization steps = 1000
 CPU: False Adam: True, Prec: no, Prior: True, Grad: True, TextTr: True
 Allocated: 4.0GB
 Reserved: 4.1GB

Steps:   0%|                                                                                  | 0/1000 [00:00<?, ?it/s]Error no kernel image is available for execution on the device at line 167 in file D:\ai\tool\bitsandbytes\csrc\ops.cu
Traceback (most recent call last):
  File "C:\Python\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\SD\stable-diffusion-webui\venv\Scripts\accelerate.exe\__main__.py", line 7, in <module>
  File "C:\SD\stable-diffusion-webui\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 43, in main
    args.func(args)
  File "C:\SD\stable-diffusion-webui\venv\lib\site-packages\accelerate\commands\launch.py", line 837, in launch_command
    simple_launcher(args)
  File "C:\SD\stable-diffusion-webui\venv\lib\site-packages\accelerate\commands\launch.py", line 354, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\\SD\\stable-diffusion-webui\\venv\\Scripts\\python.exe', '.\\launch.py']' returned non-zero exit status 1.

@ZeroCool22
Copy link

Same error, we will get a solution for this?

@kokakin
Copy link

kokakin commented Nov 10, 2022

Same error training with CPU, it occurs on both my laptops after the third step :(

Error:

Starting Dreambooth training...
VRAM cleared.
Allocated: 0.0GB
Reserved: 0.0GB

Loaded model.
Allocated: 0.0GB
Reserved: 0.0GB

Scheduler Loaded
Allocated: 0.0GB
Reserved: 0.0GB

Steps: 0%| | 3/2100 [02:44<31:59:29, 54.92s/it, loss=0.0215, lr=5e-6]Error no kernel image is available for execution on the device at line 167 in file D:\ai\tool\bitsandbytes\csrc\ops.cu

@kokakin
Copy link

kokakin commented Nov 10, 2022

unchecking "Use 8bit Adam" solved it for me!

@d8ahazard
Copy link
Owner

Closing and linking to related discussion:

#99

@si60
Copy link

si60 commented Mar 4, 2023

unchecking "Use 8bit Adam" solved it for me!

Please tell me where I can uncheck that.

@kokakin
Copy link

kokakin commented Mar 4, 2023

unchecking "Use 8bit Adam" solved it for me!

Please tell me where I can uncheck that.

It might have changed from last time I've pulled but it was in the same tab where you check "CPU only", sorry I can't check for you since I began using a different tool that doesnt really like this extension.

@si60
Copy link

si60 commented Mar 4, 2023

Thank you for kindly telling me. If you don't mind, could you tell me what tools you are currently using?

@kokakin
Copy link

kokakin commented Mar 4, 2023

Thank you for kindly telling me. If you don't mind, could you tell me what tools you are currently using?

No problem :)
I don't think I can link it to you but TheLastBen has done a colab for dreambooth, and also a colab for the automatic UI. The latter has the same tools you would use normally in the UI which implies that this extension "should" work in the colab environment, but I haven't tested it properly, so you will have to try it to see if it works.

I think this is offtopic with the issue above, so if you have further questions I would suggest going to the discussions tab, for the sake of both of us and so we don't annoy everyone who's subscribed to this discussion. Thanks for understanding.

@SystemAvinco
Copy link

unchecking "Use 8bit Adam" solved it for me!

Please tell me where I can uncheck that.

The entry is old, but if someone still has the problem. The "AdamW8Bit" value can be changed to "AdamW" in the "Optimizer" field. That solved the problem for me. Thanks to kokakin for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants