Long-term plans for growing list of EXPORT_SYMBOL_GPL

[jittygitty]

@ryao This is continuation from #14555

I was also wondering; would it be possible and any good advantages for zfs to be able to use the kernel's page cache? (some criticism can be found over at https://storytime.ivysaur.me/posts/why-not-zfs/ )

(Update, for reference: #13689 (comment) )

So how will/should zfs deal with ever growing list of export symbols being made GPL only?

[RAMChYLD]

Seconded, would like to see something being done for this. Arch updated it's zen kernel to 6.2.8 and suddenly there are a bunch of symbols that causes zfs modules to not compile.

[KungFuJesus]

Yikes.
In ZFS you can use zpool clear to roll back to the last good snapshot, that’s better than nothing.

Sigh.  This was clearly written by a user who has never used ZFS personally.  A lot of the cons on this list are or will be mitigated by pending planned changes.  The ARC contention is definitely not well understood by non-habitual ZFS users.  ARC can and often is clamped to a max value on systems that absolutely need it.  And while there have been some cases of that limit being violated at certain points in history, it's not the pain point a lot of naysayers make it out to be.  Now, the page cache fighting with ARC and causing memory starvation for one or another: that is a potential issue, though mostly a Linux specific one.   The page cache size is also tunable, though.

The dedupe table consuming too much memory has long been mitigated by special vdevs (and prior to that, could be somewhat helped by having a larger L2ARC).

The reflink copy gripe is already being addressed by a very recent PR that just went in.

The layering violation is a strength rather than a weakness. The "layering" is a dumb abstraction that grew from volume management from AIX and other ilk (Sun's SVM, etc). Having a file system aware of which blocks are backing what and a robust checksum allows for better error recovery scenarios and visibility to where the corruption lives.

whereas a file-based RAID has to perform a lot of random seeks

What on Earth is this guy talking about? The distinction of a "file RAID" makes no sense in this context.

In spite of all of the flaws in this blog post, the point that does need to be contended is that Linux kernel maintainers are constantly hiding previously exported symbols behind GPL protection for very unclear reasons. I would like to know the opinion of high level maintainers that are working on ZFS specifically for Linux professionally on this and what can be done to make the situation better. Surely LLNL and other national labs are getting pretty fed up with this?

[jittygitty]

@KungFuJesus Sorry but you missed the point of this discussion, it's only about how to deal with increasing number of changes from EXPORT_SYMBOL to EXPORT_SYMBOL_GPL. The "ONLY" reason I included that link was specifically to ask @ryao if it "would it be possible and any good advantages for zfs to be able to use the kernel's page cache..."

I had actually started to write a "rebuttal" to that very "subjective" anti-ZFS article, but that's another story and not the topic for which I opened this discussion. But here's a preview of a point from my draft rebuttal that I had started "some time ago" but never finished:

"rebuttal exposing the dishonesty of: https://storytime.ivysaur.me/posts/why-not-zfs/ where he shows none of the weaknesses of btrfs even after recommending bcachefs, which has even today at TOP in BOLD BIG letters: "The COW filesystem for Linux that won't eat your data." And OBVIOUSLY it is making fun of BTRFS and its reputation for having eaten people's DATA, there's no other COW filesystem for Linux that it could be referring to other than BTRFS! And he keeps comparing ext4 to ZFS though ext4 has none of the ZFS features, when he compares performance etc, and he shields and protects btrfs from same criticisms and only brings up BTRFS to mention some feature it has that ZFS doesn't but never the other way around. It's very dishonest and subjective while 'pretending' to be an objective analysis..."

So yes, I agree with you that the article is not a good valid critique of ZFS, but my specific question about using the Linux page cache still stands, look forward to an answer from someone knowledgeable on that.

And main point of discussion is as title says: "Long-term plans for growing list of export_symbol_gpl".
Sorry if you misunderstood me as somehow siding with the anti-zfs rant in that article, far from it.

To answer your last question, if you read a lot of the linux kernel emails, you'll see they did plan on making more and more symbols to be GPL-only. I can only hope that it is all genuinely just to hurt "closed source" proprietary kernel modules, ie nvidia etc. Because if that's the case then I see some hope that we can use all the symbols ourselves being copyleft/CDDL and still stay on good terms with them.

Also, I think a lot of people, even organizations internally might already be using all the symbols via patch and just not saying anything publicly about it. I'm in favor of considering all export symbols to be "practically" Export_Symbol_Copyleft, and build ZFS as GPL binary by changing CDDL to GPL in META file.

[YaroKasear]

I didn't read the article, but it reminds me a lot of the sort of stuff Linus Torvalds had said about ZFS. Most of what he had to say about ZFS showed he had a really lousy understanding about anything to do with ZFS. He just knows Oracle is steward of the original codebase, therefore it's bad. His rant at the time seemed almost like he didn't know the difference between OpenZFS and Oracle ZFS. It was also ironic that he dismissed so much of ZFS's feature set while putting his own personal blessing on btrfs and bcachefs, two filesystems that are more or less "cloning" ZFS-like functionality.

Goes to show: You can be an avid Linux user like me and not take everything Linus says as gospel. He's not correct nearly as often as he thinks he is. Even if OpenZFS were somehow able to relicense under the GPL I could still see Linus just rejecting the filesystem from being mainlined anyway just because it's obvious he just doesn't like ZFS, probably because "anything touched by Oracle is poison." Except apparently OpenJDK, MariaDB, LibreOffice... You know, forks of code Oracle owns. People have this weird tunnel vision when it comes to ZFS that they don't have with other projects that exist to get away from Oracle's control.

To go back to the relevant part of the discussion, though: I, too, would like to see a long-term plan to tackle EXPORT_SYMBOL_GPL. Philosophically speaking it actually flies in the face of the GPL given it only stops things like DKMS from working properly on grounds that shouldn't matter at that point. The GPL only applies to redistribution, so there's no reason to block non-GPL code from touching the kernel the way it does.

It honestly feels like the only reason EXPORT_SYMBOL_GPL is spite for out-of-tree modules altogether. The kernel documentation already makes it painfully clear there's a general hostility toward things not already in the kernel tree.

[eirnym]

Speaking about "Oracle touch", there's two things here. First is what the license say explicitly, second is the project has been forked and from which code base.

As far as I remember, ZFS in FreeBSD has been forked just before Oracle bought Sun, the original author of ZFS code.

[YaroKasear]

Indeed it had. Though I think it wouldn't have mattered if it was after Oracle acquired Sun, either. The CDDL was still effective for the versions of ZFS before Oracle made it proprietary, which makes fears about what Oracle might do all the more unfounded.

Oracle deserves its rotten reputation, but if I had one single criticism of my fellow Linux user, it's probably blind paranoia about what evil greedy corporations think about Linux or open source software. If Oracle wanted to sue over ZFS they'd have done it a long time ago when companies like ixSystems started making money off the use of OpenZFS or Apple when they were originally planning to use OpenZFS.

[stalkerg]

If Oracle wanted to sue over ZFS they'd have done it a long time ago when companies like ixSystems started making money off the use of OpenZFS or Apple when they were originally planning to use OpenZFS.

it just means they have no profit from that action right now you can remember the Android case. It is still a big issue. I suppose investing in btrfs or bcachefs will be much better.

[jittygitty]

@YaroKasear @eirnym @stalkerg Unfortunately there has been a lot of misinformation on the ZFS CDDL license chosen by Sun way back. I've looked into this quite a lot, I had started a "draft" issue on here to try and explain the history and actual reasons behind license choice to dispel the insidious conspiracy theories making Sun out to be as bad as Oracle. You can read the draft here: #13415 but note draft was never finished due to discomfort with the topic and impatience. To summarize, Sun was a great company, we owe a lot to them, and the truth is that the GPL is not the greatest copyleft license, you can actually be more likely to get sued with GPL, than CDDL, due to lack of "patent grant".

What people don't realize, is whatever perceived insult from Linux kernel developers or even Linus himself at times, against the CDDL, the exact "same" applies to GPLv3! Try to let that "sink in" before getting all confused that CDDL is 'incompatible'.

As long as Oracle hasn't retired their closed-source ZFS, they will do what they can to hurt OpenZFS integration into Linux.

(Take a look at: https://www.patreon.com/posts/33178775
Unfortunately, his "conclusion" is flat-out wrong, the only "solution" is NOT to make ZFS or OpenZFS into GPL, in fact you would have to make it all only GPL"v2" if you did, since GPLv3 is not a tittle more "compatible" to GPLv2 than CDDL itself! And what I've been trying to "enlighten" people about is that we do NOT need for every "COPYLEFT" project to all use just "GPLv2" in order to play fine with Linux. In fact, frankly, I think that's a terrible idea. There is no "viable" legal challenge that Oracle can make against Linux distributions using OpenZFS, that is the reality which Oracle actually doesn't seem to "like".)

The "reason" that Oracle is "silent", is because they are using their "silence" as a "big bluff", to instill "fear", and sadly it appears to have worked even on Linus to some extent. The "reality" is, that OpenZFS with it's CDDL license can live long and prosperous with Linux without any license change. For all practical intents and purposes, OpenZFS project is "just as" copyleft as the Linux kernel and its GPLv2 license.

I've always said the best solution is for Linus to accept that GPL export symbols should be treated as "Copyleft Exports" instead of "GPLv2" exports which are only to be used by GPL "v2", because GPLv3 code is in the "exact same boat" as CDDL.

Or Linus and the Kernel devs can easily say: "Hey guys, since you/OpenZFS are literally fully Open Source and copyleft just like us, go ahead and use a "Condom" and we can hook up no problem anytime!"

In this way we/zfs can go ahead and continue modularizing and include more and more code into our current GPL shim while allowing it to freely use any "GPL" labelled export symbols. Currently I think that even though we/zfs uses a GPL shim https://github.com/openzfs/spl that the decision had been made to "still avoid" using GPL export symbols even though this GPL shim module is the way that actual "CLOSED SOURCE" (not open source at all imagine that) have been in use by Linux for "ages", and apparently without anyone on the Linux side doing anything about it legally, regardless of any "talk" etc. If Linux knows it can't win court cases against closed source using "condom" shims, then they surely know they will never win a court case against open-source copyleft code using condoms/shims when their "copyleft intents" have been practically met.

So if closed source software can live with Linux, how much more so can OPENzfs. Point is that I think Linus would/should not have any problem with OpenZFS using Linux kernel "GPL" export symbols, all our code is fully "open source" and "copyleft". His own stated "intent" (copyleft) for when he stumbled on GPL long ago and chose it, is not interfered by OpenZFS at all.
 
I do NOT recommend this "condom" solution which all the CLOSED-Source kernel modules use. The real solution is changing "GPL exports" to "Copyleft Exports". Because in fact the reality is that "EXPORT_SYMBOL_GPL" is a "lie" as it is anyway, it is NOT "EXPORT_SYMBOL_GPL" but in fact it is "EXPORT_SYMBOL_GPLv2", since GPLv3 isn't at all more compatible than CDDL.

Anyway, main point is Oracle can't do nothing about it, all they do is fearmongering "bluffs" which too many have fallen for.

And until Linus decides to give his official blessing for GPL exports to be considered as "Copyleft exports", anyone is 100% free to do so already when "building" ZFS, and distributions can provide a "script" to enable end users to easily do so also. 

[YaroKasear]

I'm definitely in agreement with you.

I'm pretty strongly against the EXPORT_GPL_SYMBOL practice largely due to how it's not really making anything really "compliant" with the GPL anyway.

Take the fact that the GPL, regardless of version, only puts terms and conditions on distribution of software. It's been stated time and time and time again by the FSF and others that as far as the GPL is concerned end users can do whatever they want, even "GPL violating" things, and that the "you must make your changes open" part only applies when you distribute the software. And this probably only applies to EXTERNAL distribution, at that. I doubt it's a violation for, say, Google to modify the Linux kernel for internal-only changes and not make those changes available to the public since they don't plan to distribute the modified kernel into the public sphere. Maybe the source would have to be made available internally at Google, but I doubt the GPL would require that.

Why I bring this up is that the GPL-only symbol export thing only seems to put a restriction on end users who the GPL wouldn't have anything to say about how they use things like the kernel. Maybe they could make an argument for distributing already-compiled modules, but the kernel license already EXPLICITLY makes an exception about the whole "derivative works" issue by saying even proprietary code can link to kernel symbols, making the GPL-only symbols even more meaningless and seemingly vitriolic.

I don't trust Oracle at all, but I definitely don't think they're going after OpenZFS. Yeah, they went after Google over Java and that didn't entirely work out, and their basis for going after things like OpenZFS would be even shakier based on how open source licenses work in the first place. It's not helped that by this point OpenZFS and Oracle ZFS are very different codebases now.

Also, Oracle's had much more popular, juicier targets like LibreOffice, MariaDB, etc. that Oracle would likely get more money for pursuing than OpenZFS... yet for some reason the people FUDding ZFS on Linux seem not to bat an eyelash on those projects despite identical origins to OpenZFS.

If you're going to spread fear of using OpenZFS, then go full-hog and FUD MariaDB and LibreOffice. They're also open source Sun properties with Oracle copyright stewardship like OpenZFS is.

The reason they don't is because Linus didn't blast them at all, the way he blasted ZFS despite making it clear he:

1. Never used OpenZFS before.
2. Doesn't know the difference between OpenZFS and Oracle ZFS.
3. Doesn't know anything about OpenZFS or ZFS in general.

Bonus points for calling it a "buzzword" when he's allowed btrfs on the kernel for a long time now... a filesystem effectively trying to "reinvent" the "buzzword" for the Linux kernel.

Linus isn't always right.

Amusingly, at that point he also blasted OpenZFS for being under active development or something like that. Looking at btrfs that felt like hypocrisy to me, as btrfs has been around since 2008 and has yet to reach the maturity or stability of something like OpenZFS or even non-CoW Linux filesystems like ext4.

I've written off btrfs as a lost cause for serious storage a long time ago. I've still had it completely wreck my data on a single drive formatted with it. Forget the RAID 5 write hole: btrfs isn't safe for single disk usage as far as I'm concerned.

I would love to see bcachefs go somewhere, but with only one author behind it and it making little to no progress making it into the kernel, I'm not holding my breath.

That said, I'm not really expecting ZFS to make it into the kernel, nor do I really want it or not want it to. Linus already made it clear it won't even if there wasn't any issue with the license.

Sun was a fantastic company, especially as they started opening up their code and making things like OpenZFS possible. Linus can have OpenZFS when he can pry it out of my cold, dead fingers, since there's no filesystem "native" to Linux that is even comparable.

[jittygitty]

@YaroKasear Thanks I can see by the points you made that you're definitely a lot more knowledgeable on the license issue than most of others whose lack of knowledge perhaps leads to their fudding and fearmongering, so I have "hope" for them.

If I didn't have such hope to be able to change and convince others, I wouldn't have wasted my time trying to explain things.

You made a huge point by the way, which is that Linus himself is on record (in writing also), that as you said, "even" closed source kernel modules can link to the kernel without becoming "derivative works", hence in that way they are "compatible", simply if the closed-source module had not been developed specifically "for Linux", he gave example of "ported" modules.

And if he admits that for "closed source" modules, he would/should have no problem with a fully "open source" module like O-zfs which not only also more than meets all his criteria of not being a derivative work, but even his "intent" of "copyleft" : )

Anyway, though Linus isn't a lawyer and has said some inaccurate things at times perhaps, I still think that he's not really an enemy of openzfs. I actually think he's more open minded than others in the Linux kernel, so I have "hope he can be an ally".

And as far as Oracle, yep, they "would" do anything to hurt their competition, if they have not, it's only because they can't : )