Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prototype to generate COPYRIGHT from REUSE metadata #104439

Merged
merged 6 commits into from
Dec 6, 2022

Conversation

pietroalbini
Copy link
Member

This PR adds a prototype to generate the COPYRIGHT file from the metadata gathered with REUSE. There are two new tools:

  • src/tools/collect-license-metadata invokes REUSE, parses its output and stores a concise JSON representation of the metadata in src/etc/license-metadata.json.
  • src/tools/generate-copyright parses the metadata generated above, (in the future will) gather crate dependencies metadata, and renders the COPYRIGHT.md file.

Note that since the contents of those files are currently incorrect, rather than outputting in the paths above, the files will be stored in build/ and not committed. This will be changed once we're confident about the metadata.

Eventually, src/etc/license-metadata.json will be committed into the repository and verified to be up to date by CI (similar to our GitHub Actions configuration), to avoid having people install REUSE on their local machine in most cases.

You can see the (incorrect) generated files in https://gist.github.com/pietroalbini/3f3f22b6f9cc8533abf7494b6a50cf97.

r? @pnkfelix

@rustbot rustbot added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 15, 2022
@rust-log-analyzer

This comment was marked as resolved.

@pietroalbini pietroalbini marked this pull request as draft November 15, 2022 12:31
@pietroalbini pietroalbini marked this pull request as ready for review November 15, 2022 14:15
@pietroalbini
Copy link
Member Author

Fixed the tidy failure and revamped the Markdown output to actually be readable. This is ready for review.

config.toml.example Outdated Show resolved Hide resolved

let mut result = Vec::new();
for file in document.file_information {
let license = interner.intern(License {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as someone unfamiliar with SPDX and with the overall plans here: What is the expected interning behavior when the file has differing copyright_text but the same concluded_license ? Are those supposed to be interned to the same LicenseId, or to distinct LicenseId's?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same license but different copyright holder would result in different LicenseIds, as that's effectively a different license (since most licenses require attribution, and the copyright statement is the attribution).

@pnkfelix
Copy link
Member

@pietroalbini do you have any thoughts on how to encode some tests for this prototype? E.g. some sample files and maybe a mocked up reuse binary that does something trivial to extract copyright/licensing info?

I won't block landing a prototype here on the presence of such tests, but it is something worth a little bit of thought. (e.g. Cargo already uses some solution to automatically generate directory/file structures to use to drive its tests; is there work there that could be leveraged here, rather than making something from scratch?)

Co-authored-by: Felix S Klock II <[email protected]>
@pietroalbini
Copy link
Member Author

Yeah I'll add tests to a followup PR (added it in my todo list).

@pietroalbini
Copy link
Member Author

Addressed all feedback.

) -> Node<&License> {
match node {
Node::Root { childs } => Node::Root {
childs: childs.into_iter().map(|child| strip_interning(child, interner)).collect(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... how is this code still building, given that there are references to the old name in this recursive call here that were not updated as part of the rename to expand_interned_licenses?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, that's because it's not building anymore. Whoops, good catch.

@pnkfelix
Copy link
Member

pnkfelix commented Dec 3, 2022

( r=me once @pietroalbini addresses my concern about my hypothesized-to-be-broken alpha renaming. )

@pietroalbini
Copy link
Member Author

@bors r=pnkfelix

@bors
Copy link
Contributor

bors commented Dec 6, 2022

📌 Commit f8a7123 has been approved by pnkfelix

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 6, 2022
@pietroalbini
Copy link
Member Author

@bors rollup

bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 6, 2022
…iaskrgr

Rollup of 11 pull requests

Successful merges:

 - rust-lang#104439 (Add prototype to generate `COPYRIGHT` from REUSE metadata)
 - rust-lang#105005 (On E0195 point at where clause lifetime bounds)
 - rust-lang#105098 (propagate the error from parsing enum variant to the parser and emit out)
 - rust-lang#105243 (remove no-op 'let _ = ')
 - rust-lang#105254 (Recurse into nested impl-trait when computing variance.)
 - rust-lang#105287 (Synthesize substitutions for bad auto traits in dyn types)
 - rust-lang#105310 (Be more careful about unresolved exprs in suggestion)
 - rust-lang#105318 (Make `get_impl_future_output_ty` work with AFIT)
 - rust-lang#105339 (support `ConstKind::Expr` in `is_const_evaluatable` and `WfPredicates::compute`)
 - rust-lang#105340 (Avoid ICE by accounting for missing type)
 - rust-lang#105342 (Make `note_obligation_cause_code`  take a `impl ToPredicate` for predicate)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit e5a01b9 into rust-lang:master Dec 6, 2022
@rustbot rustbot added this to the 1.67.0 milestone Dec 6, 2022
@pietroalbini pietroalbini deleted the pa-generate-copyright branch December 6, 2022 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants