Error no kernel image is available for execution on the device at line 167 in file.

[ZeroCool22]

Solution?

GPU: 1080TI

Also, dunno why it says something about "D", i have nothing installed in D.

This is the only CSRC folder i have and there is not any ops.cu file.

[hdon96]

Same issue here
#34

[ZeroCool22]

I did this, still same error...

[james-things]

I am having the same issue, also running a 1080Ti. I found some troubleshooting for this on the bitsandbytes repo which suggests either a CUDA version mismatch or a GPU incompatibility. (https://github.com/TimDettmers/bitsandbytes/blob/main/errors_and_solutions.md#No-kernel-image-available)

I believe I have now resolved all CUDA discrepancies, but I also noticed that to 10 series cards are incompatible with LLM.int8(), perhaps the source of the error is LLM.int8() being called on 1080Tis? We are likely the only people trying to run this on GPU that have that compatibility issue, with the other 10 series cards being <10GB.

More info: https://github.com/TimDettmers/bitsandbytes#requirements--installation

[tmode-1960]

I have a Titan X(pascal), same family of GPU

[hdon96]

Yes, 1080 ti here too

[d8ahazard]

Gonna close this and shift it to a discussion, as there are several related issues that all seem to be tied to this GPU family.

#99

[ZeroCool22]

Still getting same error and CUDA is updated.

[james-things]

I started digging into this a bit yesterday; I am totally unfamiliar with cuda so I could be wrong, but it seems as though the dll may need to be compiled for the card family.

This presents a challenge because the Makefile for bitsandbytes is Linux only, and the maintainer has said they do not have access to windows/would need someone else to help in porting it.

There is a user of another project who managed to compile it himself - it is on Github if you search. They also posted the changes they had to make - I am sorry for not including the link but if you Google the dll name you will find it. I am thinking that this person's github repo may be the origin of the cu116 dll we are trying to use now.

I have been through pretty exhaustive troubleshooting on this one already, so I am going to give building the dll a shot when I get the chance. No promises, but I will share the finished result and provide a how-to if it works. Should get around to it within a week, hopefully less. :)

Edit: posted this on mobile but managed to find the link nevertheless:

bitsandbytes-foundation/bitsandbytes#30 (comment)

[ZeroCool22]

I started digging into this a bit yesterday; I am totally unfamiliar with cuda so I could be wrong, but it seems as though the dll may need to be compiled for the card family.

This presents a challenge because the Makefile for bitsandbytes is Linux only, and the maintainer has said they do not have access to windows/would need someone else to help in porting it.

There is a user of another project who managed to compile it himself - it is on Github if you search. They also posted the changes they had to make - I am sorry for not including the link but if you Google the dll name you will find it. I am thinking that this person's github repo may be the origin of the cu116 dll we are trying to use now.

I have been through pretty exhaustive troubleshooting on this one already, so I am going to give building the dll a shot when I get the chance. No promises, but I will share the finished result and provide a how-to if it works. Should get around to it within a week, hopefully less. :)

Edit: posted this on mobile but managed to find the link nevertheless:

TimDettmers/bitsandbytes#30 (comment)

I replaced the DLL's but still get the error, we need to modify also some code too?

[james-things]

Did you compile the dlls yourself, or do you mean you replaced it with the ones from github?

I was saying that I suspect that maybe the dll we already had was that one from github. I didn't see the maintainer of this repo mention where it came from and I couldn't find any others while searching around. Just a guess though.

[ZeroCool22]

Did you compile the dlls yourself, or do you mean you replaced it with the ones from github?

I was saying that I suspect that maybe the dll we already had was that one from github. I didn't see the maintainer of this repo mention where it came from and I couldn't find any others while searching around. Just a guess though.

Replaced with ones on that repo.