Should PSF-2.0 be templatized further for projects?

[alandtse]

In our description of PSF-2.0 we mention that some projects have adopted it as their license.

I recently ran into a project that has made changes to the license to address their project. Should we consider templatizing the license further to recognize this?
https://cx-freeze.readthedocs.io/en/latest/license.html#license

Here's an image of the diff where you can see pretty quickly what should be optional or just replaceable.

[JimVitrano]

I'll be happy to work on this with you, Steve.

[swinslow]

Thanks @JimVitrano!

[swinslow]

I've been working on the markup for these changes. I've run into one error which, after staring at it for a while and trying several variations, I think might be an issue with the tools rather than the markup.

I'm cc'ing @goneall for us to look at this, though since I'm hoping to push 3.12 later today I'm going to shift this issue to 3.13.

In the revised markup, I'm trying out a modified test file for the relevant part of the license text from https://cx-freeze.readthedocs.io/en/latest/license.html#license. See attached for the test file I'm using, as well as the work-in-process modified template markup I've been working on (with the template filename changed from ".xml" to ".xml.txt" b/c of GitHub restrictions):

WIP template: PSF-2.0.xml.txt
test text: PSF-2.0.txt

When running the tests for these, I'm seeing the following error:

The following warning(s) were identified:
	Test for license PSF-2.0 failed: Invalid template found for license ID PSF-2.0: Invalid template rule found during compare: Unknown rule keyword: All Rights Reserved""

This appears to be based on the following chunk from the markup, since it's the only bit that has "All Rights Reserved" in it:

Subject to the terms and conditions of this License Agreement, <alt match=".*" name="copyrightHolder1">PSF</alt> hereby grant<optional>s</optional> Licensee
a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform
and/or display publicly, prepare derivative works, distribute, and otherwise use <alt match=".*" name="software1">Python</alt>
alone or in any derivative version, provided, however, that <alt match=".*" name="copyrightHolder2">PSF's</alt> License Agreement and
<alt match=".*" name="copyrightHolder3">PSF's</alt> notice of copyright<alt match=".*" name="copyrightHolder4">, i.e., "Copyright (c)
2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation; All Rights Reserved"</alt> are retained in <alt match=".*" name="software2">Python</alt> alone or in any
derivative version prepared by Licensee.

I've tried a bunch of variations, including removing the quotation marks within the copyrightHolder4 alt tag, but I can't seem to find any combination that will work without throwing this error.

@goneall -- apologies if I'm missing something obvious, but wanted to flag in case this is an edge case that's causing some issue with the tools. It looks to me like https://github.com/spdx/tools/blob/17b6f86dce22a41223fe8324e7586132bc5f5b23/src/org/spdx/licenseTemplate/LicenseTemplateRule.java#L263 is where this error is being thrown, but beyond that I'm not familiar enough with the Java tools to backtrace it any further. Grateful for any help you can provide here!

[goneall]

@swinslow I don't see anything obviously wrong - so I also suspect the tool.

I found the XML file above - I'll take a look and update the issue on what I find.

[swinslow]

Thanks @goneall! When you say the "source XML file" -- did you mean something different from what I linked in the prior comment, as an attachment? Here's the link again:

https://github.com/spdx/license-list-XML/files/6097797/PSF-2.0.xml.txt

It would just need to be renamed from .xml.txt to just .txt -- because GitHub wouldn't let me upload a file with filename .xml unfortunately...

[goneall]

It's the double quotes " that's tripping up the tools. The current publisher doesn't escape the double quotes when translating to the the template rule
<<var;name="copyrightHolder4";original=", i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006 Python Software Foundation; All Rights Reserved"";match=".*">>

You can work around this by changing the double quotes to single quotes.

I would add escaping to the publisher, but that would require a change to the spec to add escaping characters for the original=. Rather than fix the template text, I would prefer to standardize our XML format and deprecate the template format alltogether. This would allow much easier parsing and get rid of some ugly code in the process.

[goneall]

I just tested the workaround, and it doesn't like the semi-colon either. I'll update when I do a couple more tests.

[goneall]

OK - I was wrong on the initial diagnosis.

This is due to the semicolon after the Python Software Foundation;

Removing or replacing the semicolon works.

You can leave the double quotes in.

The semi-colon can be escaped with a backslash, however, I noticed that the backslash is rendered in the HTML.

See issue spdx/LicenseListPublisher#100 for more details.

[ariel11]

@swinslow - is the author and date text going to be changed to colored font under the matching guidelines? The matplotlib license is another example of what could use the SPDX "PSF-2.0," however the author and date text has been changed: https://github.com/matplotlib/matplotlib/blob/master/LICENSE/LICENSE.

[jlovejoy]

@swinslow - what is the update on this?

[jlovejoy]

review with #1200

[swinslow]

Unfortunately I'm going to roll this one forward again. I may have a solution based on part of how we handled the nested quotes in the new Python-2.0.1 markup, but I need to check the changes for this against the tool. And don't want to hold up the 3.18 release again.

I think I have a path forward and will try to make sure this gets handled in the near future...

[swinslow]

I've submitted #1844 to fix this.

To deal with the technical issue I had reported above previously, I followed @goneall's recommendation (thank you!) to just remove the semicolon. It's in replaceable copyright text, so no issues with removing it from the template.

That said, I was then running into odd problems with trying to use <optional>s</optional> markup at the ends of a few pluralized words (e.g. "grant" vs. "grants"). Whether or not spacing="none" or spacing="after" was included, it still didn't want to match against the plain text. I poked at it for quite a while, but couldn't figure out a workable fix, so instead I switched to doing <alt match="grant|grants"... and similar in a few places. That could perhaps be cleaned up down the road with <optional> tags if the issue is sorted out, but at least this will get the fix in place and let us put this one to bed.