Adding functionality for dirhash in library

[matglas]

Related to: in-toto/witness#436

This a different approach to #65. It captures a dirhash for material and product if the dir is mentioned as an argument.

witness run --dirhash-glob node_modules/* --step build -- npm clean-install

The argument can be repeated multiple time and globs with ** are not allowed.

The output will generate a dirhash based on the algorithm use by golang in https://pkg.go.dev/golang.org/x/mod/sumdb/dirhash and is the same as described in the in-toto digest spec (here)

The subject for a dirhash will be https://witness.dev/attestations/product/v0.1/dir:node_modules/example/.

The policy works correctly and also artifactsFrom are correctly checked.

Todo

• allow verify to check not only file subject but also dir subject
• decide on the copying of the glob library into the go-witness library.
• discuss what additional docs on the library would be benificial

[ChaosInTheCRD]

Hey! thanks for submitting this 😄. It looks like this is the sort of functionality we were looking for after the discussion on the community call, so thanks a lot for following up with it.

After looking through things, it seems as though everything looks good, although the use of "github.com/gobwas/glob" worries me a tad since it's a package that belongs to an inactive user and hasn't been committed to for 6 years.

It's probably a tad ironic since at this time we don't have any strong policy over the kind of packages we are and are not happy to pull in. But maybe in this case, a bit of copying might be better than depending? 🤔 Just a thought.

I would add also that we need to get tests written before this is merged, and docs updated in github.com/in-toto/witness before in-toto/witness#436 is merged.

[matglas]

Thanks for the input. What is ok OSS wise in copying code?

[adityasaky]

I haven't had a chance to look at this PR, but I wanted to check if it's consistent with the implementation in in-toto/in-toto. Is that the case? Thanks!

[matglas]

I haven't had a chance to look at this PR, but I wanted to check if it's consistent with the implementation in in-toto/in-toto. Is that the case? Thanks!

Yes as far as the output of the commandline example in the code there goes and what I saw in the digest documentation. I have not checked the code in all its detail.

[matglas]

@ChaosInTheCRD I updated and fixed the tests to do some dirhash testing.

[matglas]

Cherry-picked all commits onto latest main to fix DCO issues.

[matglas]

@jkjell all should be good to merge now. There are not breaking changes to witness. The usage will be made available in the other repo. But even with the changes in it should not break any other behavior.

[jkjell]

Great work @matglas! 🎉

[matglas]

Great! Thank you!