Module build fails on aarch64 due to using GPL-only symbol

[imrehg]

System information

Type	Version/Name
Distribution Name	Arch Linux ARM
Distribution Version	rolling
Kernel Version	6.2.1
Architecture	aarch64
OpenZFS Version	zfs-2.1.9-1

Describe the problem you're observing

Failing to compile kernel module for zfs.

Describe how to reproduce the problem

Try building the zfs dkms module on the device.

Include any warning/errors/backtraces from the system logs

$ sudo dkms autoinstall
[...snip...]
  MODPOST /var/lib/dkms/zfs/2.1.9/build/module/Module.symvers
ERROR: modpost: GPL-incompatible module zcommon.ko uses GPL-only symbol 'kernel_neon_begin'
ERROR: modpost: GPL-incompatible module zcommon.ko uses GPL-only symbol 'kernel_neon_end'
ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'kernel_neon_end'
ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'kernel_neon_begin'
make[2]: *** [scripts/Makefile.modpost:138: /var/lib/dkms/zfs/2.1.9/build/module/Module.symvers] Error 1
make[1]: *** [Makefile:1973: modpost] Error 2
make[1]: Leaving directory '/usr/lib/modules/6.2.1-1-aarch64-ARCH/build'
make: *** [Makefile:55: modules-Linux] Error 2
make: Leaving directory '/var/lib/dkms/zfs/2.1.9/build/module'

[rincebrain]

Yes, it appears they broke it deliberately in torvalds/linux@aaeca98, first in 6.2.

Given the limited existing deployment of non-GPL modules for arm64 and the
fact that other architectures like x86 already make their equivalent
functions GPL only this is not expected to be disruptive to existing users.

Uh-huh.

[imrehg]

Good find, cheers! Well, this is a bummer, I guess a kernel downgrade for the time being and then we'll see...

Anything you suggest that I can help, or comment, put in a word as support anywhere?

[ryao]

You could email Mark Brown <[email protected]> and Will Deacon <[email protected]> telling them that they were wrong about the change not being "disruptive to existing users". Those symbols are, at their core, CPU instructions and marking them GPL exported just means that those that want to use FP/SIMD inside the kernel needs to call those instructions themselves. EXPORT_SYMBOL_GPL() is not meant to bar non-GPL modules from accessing trivial helper functions.

As for what x86 does, it originally had been EXPORT_SYMBOL(), new functionally identical symbols with EXPORT_SYMBOL_GPL() markings were introduced, everything in the tree was switched to them and then the original symbols were dropped. When end users complained, people simply cited the kernel policy about not removing symbols not being used inside the tree, even though it had effectively been changing a pair of symbols that were long under EXPORT_SYMBOL() to EXPORT_SYMBOL_GPL(). The maintainers refused to restore access to the original functionality by putting the new symbols under EXPORT_SYMBOL(), even though the symbols implemented trivial functionality that anyone could use merely by using the correct CPU instructions. We eventually did that, but it took time to do.

Other architectures are not under any obligation to follow x86's example and it would be appreciated if it would not, since even if it is trivial, this forces unnecessary duplication of effort and we are already busy working on improvements for users. If the past x86 fiasco is any indication, it will take months for us to get to it and it would spare end users pain during that time if mainline reverted this change.

[microcai]

time to license openzfs as GPL !

[rincebrain]

You understand that a lot of OpenZFS's codebase is owned by Oracle, right, and can't be relicensed by anyone working on it?

[ryao]

@microcai The OpenZFS codebase is owned by hundreds of stakeholders that would all need to sign off on a license change. As a consequence, doing a license change is impossible.

That said, many stakeholders would be against a change to the GPL 2.0 (or any version of the GPL for that matter). Going with BSD-2 would be far more likely, but the CDDL has the benefit of a patent grant, so this likely would be unrealistic while there are patents held by Oracle and IBM that apply to the OpenZFS codebase. In particular, Sun licensed the ARC algorithm from IBM for use in ZFS, and we have a patent grant through that.

Having the Linux kernel relicensed would be more realistic. Linus Torvalds has enough influence among Linux copyright holders that he could likely get a license change done if he really wanted to do it.

[eirnym]

I'm sorry for an angry comment, but why not to put all the symbols under GPL in Linux kernel? This will solve all Linux problems by burying the Linux completely for those who want to use non-GPL licence in drivers.

[jittygitty]

@microcai > time to license openzfs as GPL !

No, not at all! Time to just BUILD ZFS as if it is GPL, we have every right to do that:
#11357

Far as I'm concerned this should be promoted in our documentation and any/all GPL-ONLY exports that can be used, should be used when we do that. If you are not "distributing", you can do whatever you want on your own machine. Have fun!

[jittygitty]

@eirnym I would prefer we keep license discussions separate from bug reports etc (but those I opened were "closed"), so to answer your question, anyone can still use even closed source proprietary drivers with Linux even if all exports are GPL_ONLY, and the GPL protects even that right.

You are not supposed to "distribute" it mixed, but you can mix Linux with "whatever" you want, the end user is the KING according to GPL philosophy.

Ironically, the Linux Kernel folks aren't doing any damage to "proprietary" closed-source software, they just use a "CONDOM" ie shim GPL driver that sits in-between their proprietary driver and the Linux kernel. Such "condom" usage is unstoppable.

Kernel devs have complained of condoms, but I've never seen them do anything about it "legally" in the courts. Perhaps because they feared the result if they lost?

Even if they won, at most they'd win against distribution of systems that come with Linux and proprietary drivers (GPU?) and their condoms, "shipped/distributed" together. They can NEVER stop any user from using any condoms and GPU or other proprietary drivers they want, so it's sad that they hurt open-source projects more.

The good news is that if you BUILD your own ZFS modules you can simply change CDDL to GPL (as per #11357 ) and be done with it. We don't need no "condoms" : )

[jittygitty]

@ryao "Having the Linux kernel relicensed would be more realistic. Linus Torvalds has enough influence among Linux copyright holders that he could likely get a license change done if he really wanted to do it." -ryao

Although I myself had not too long ago started writing a generic loosely COPYLEFT LICENSE that could bring blessed matrimony between Linux and other Copyleft projects, it would not be necessary for Linux to change its license to a more generic copyleft license, though that'd be great. Here's a SIMPLE easy fix for Symbol issues:

Ask Linus Torvalds to simply consider changing:
"EXPORT_SYMBOL_GPL" to "EXPORT_SYMBOL_COPYLEFT".

I have joked about making such a pull request before, see #11357 (comment)

Then under "COPYLEFT" the Linux Kernel could accept weak copyleft licenses like CDDL and many other copyleft licenses (even GPLv3!) as "copyleft" compliant.

Think about it, currently, GPLv3 kernel modules are just as INCOMPATIBLE (distribution-wise and Export-Symbol-wise) with the Linux Kernel as PROPRIETARY "closed source" kernel modules/drivers (GPU etc). Copyleft CDDL is in same boat.

Why should LINUS do this? To prove that what he does aligns with the philosophy he claims to have been holding to! He has always said he just wants anyone using and modifying his code to be released in source code form, he was very vocal against changing Linux Kernel from GPLv2 to GPLv3 because he had NO PROBLEM with "Tivoization". But with change I proposed, he could:

1. Keep Kernel License in his preferred GPLv2 license, tivoization compatible.
2. Make GPLv3 modules and other copyleft (CDDL/ZFS) able to use ALL SYMBOLS.
3. Make a "much stronger" case against PROPRIETARY "source-closed" drivers.

So since "EXPORT_SYMBOL_GPL" was not supposed to hurt other OPEN-SOURCE (even copyleft) projects, but was always by his own claims to be against "closed-source" drivers, what better way to prove this than EXPORT_SYMBOL_COPYLEFT?

[ryao]

@jittygitty While this is an interesting conversation, I have two thoughts:

1. This is perhaps not an appropriate forum for it. A discussion in the discussions tab might be a better place: https://github.com/openzfs/zfs/discussions/new?category=general
2. I suspect that you will get push back from mainline on this because the GPL symbols are marked such under a theory that anything using them is probably a derived work of Linux and must be subject to the GPLv2. That is not really true, but it is a line that they have drawn that permits development to proceed without the distraction of a constant debate over this. Your proposal is likely to reignite that debate, so be prepared for that.

That said, I would prefer that this issue stay on the topic of how we will handle this latest compatibility issue. If an email has not been sent to the Linux ARM maintainers yet, please include me on CC in that email. You can find my email in my github profile.

[eirnym]

@jittygitty Thank you for a long answer with a good explanation. it is a useful for those who don't know. My comment was mostly rhetorical, as I'm personally aware about this problems and solutions.

[jittygitty]

@ryao I haven't done directly or indirectly what I had "joked" about because it's NOT the best way to do it. Perhaps I should put a warning here the only/best way to get some love for all copyleft (like ZFS) from Linux and more specifically LINUS:

Linus is more understanding and more likely to be conciliatory than some of the newer kernel devs or even his right-hand team who might out of reflex jump and say something negative that will make progress difficult without losing face etc.

So, if anyone wants to propose anything similar to what I proposed, you MUST contact LINUS in PRIVATE and ideally he should KNOW YOU or at least know of you and already have "some" professional respect for you. Then you can propose to him something in-line to what I said, to change symbols so that he allows for more loosely COPYLEFT licenses like ZFS since it still lives up to his claimed purposes throughout his career and would do so without hurting other copyleft projects.

IF you can do the above in private, and then agree you will then follow-up in PUBLIC via a medium/LWN.net/Kernel/etc I think you may win LINUS over, since I don't think he wants to look like a hypocrite, and I think he genuinely doesn't want to hurt other open-source projects. BUT if you don't go through him/Linus first, privately, its going to be extremely difficult if his right-hand team does some knee-jerk response, because he can't then publicly contradict them and mess up his relationship with them, which is of higher importance to him. That's my advice.

If LINUS comes out with an extended hand in good consideration of such proposal and the conversation is made public, his right-hand team and kernel devs will be respectful of his direction and I think they will become friendlier to the idea as well.

After all, I don't think anyone has really proposed to them what I proposed, which has no compromise on their preferred license but brings harmony with all copyleft.
So, if they actually pause and think about it and realize the benefits, there's hope :)

And ryao like I said, this license discussion should be in another thread, but ask those who "closed" my other 'license' threads. (To kinda well "bury" open talk?)

Also, since this specific issue is a "victim" of these licensing issues, its pertinent.

[jittygitty]

I forgot to answer: "...how we will handle this latest compatibility issue"? Are you ok with what I already proposed? Change CDDL to GPL and build. Simplest fix, isn't it?

I don't see any problem with that myself. I'm not saying we can't "also" try to do some other more complicated workaround (but potential regressions again etc?)

[mabod]

I tested this and it works fine. No message like "module license 'CDDL' taints kernel." anymore in the journal.
I added a patch to my zfs-dkms PKGBUILD which replaces CDDL with GPL in the META file. That was it.

[imrehg]

@ryao I've followed your recommendation and sent an email to the developers of that patch, let's see what we hear back. Apologies that it took this long. I do agree with you that either a revert in the kernel (longshot) or significant OpenZFS development effort (long time) are proper solutions. Thanks for the additional context in your comment above, that helped to frame things better.

[eirnym]

@jittygitty @ryao would double-licensed kernel communication part helps. Or this is a CONDOM mentioned above and it's not strictly legal?

[mabod]

I have a question for the experts.

Someone can patch the kernel to replace EXPORT_SYMBOL_GPL with EXPORT_SYMBOL. What happens if this person distributes this patched kernel as binary and source code? My understanding is, that this does not violate the GPL. It is a simple code change but not a license change.

The intentions, the morale, the philosophy behind the existence of EXPORT_SYMBOL_GPL doesn't matter from a legal point of view. Taking out EXPORT_SYMBOL_GPL might hurt the feelings of some kernel developers. But would it have legal implications? I believe not. The only thing that matters from a legal point of view is the GPL. And that is not violate with such a patched kernel.

EDIT:
linux-tkg provided such a patch in the past to get zfs going:
https://github.com/Frogging-Family/linux-tkg/blob/master/linux-tkg-patches/5.10/0011-ZFS-fix.patch

[eirnym]

@mabod would a "statistical" user patch every kernel in every distribution just for ZFS usage?

Gentoo distribution is more patch-friendly than Debian, Ubuntu, Fedora or RHEL

[mabod]

First of all, I am just thinking out loud here. The idea would be to provide a binary kernel for a given distro. And creating the patched source code is fairly easy by scripting an sed -i 's/EXPORT_SYMBOL_GPL/EXPORT_SYMBOL/g' for all files in the source tree.

[eirnym]

@mabod found an old discussion on that: https://lwn.net/Articles/154602/

[jittygitty]

@mabod Yes you are right that "anything" inside the code of Linux kernel can be changed and even distributed since it is GPL after all. But PLEASE do not go down that path, it would create un-needed animosity. Stick to CDDL to GPL in META file. (The other method has bad repercussions and helps "proprietary closed source".)

(UPDATE, I gave further reasons on why one method is better than the other at: #13689 (comment) )

@eirnym Yes and as you also pointed out about usual/statistical Linux user..., we can much simpler simply ;) give them option to run a "patch" at build to change ZFS CDDL to "GPL" and then build as mabod already confirmed worked just fine.

I had opened a "discussion" for this export symbol topic at: #14630

@eirnym Sorry I didn't understand your question about "would double-licensed kernel communication part help"...maybe you can clarify in discussion link above.

I elaborated more to ryao on this licensing topic in a private email.
@ryao Did you get my email?