Add gpg support and custom (sub)process module

[lukpueh]

Fixes issue #:

Supersedes #163
Closes #55 and in-toto/in-toto#275

Description of the changes being introduced by the pull request:

This PR adds a Python interface for GnuPG providing functions for:

• signature creation (using private keys in a local gpg keyring)
• public key export (from a local gpg keyring into a pythonic format),
• gpg signature verification function that uses the cryptography library and does not require the gpg command line tool

Additionally, this PR adds a:

• custom (sub)process convenience module, to i.a. hide Python version compatibility details

Both, the gpg sub-package and the process module, were originally developed as part of the in-toto project, and were transferred at in-toto@9c374a9c970f0e8d985cbf06a7e6c7a7df3017c3, preserving parts of the original git history. See commit messages of 62ba239 and de817db for git-technical details about `.

For review only consider commits in the range of lukpueh/securesystemslib@47a9d9a...lukpueh:add-gpg, earlier commits have already been reviewed as part of the in-toto project.

This PR further:

• drops Python 3.4 support in favor of the process module (9116205),
• lowers the test code coverage threshold to 99%, due to a hard-to-test gpg code branch (a79667a),
• fixes an unrelated docs bug that was introduced in Expect bytes in create_ and verify_signature; do not serialize #162 (ad996ea)

DISCLAIMER: This PR does not integrate the public gpg functions with securesystemslib's public function modules, i.e. keys or interface.
This also means, that the generic keys.verify_signature function can not be used to verify gpg signatures (use gpg.functions.verify_signature instead).

While such an integration seems desirable, I suggest to do this in a separate PR, e.g. in the course of a general API revision (see #35).

• The code follows the Code Style Guidelines
• Tests have been added for the bug fix or new feature
• Docs have been added for the bug fix or new feature

[coveralls]

Coverage decreased (-0.09%) to 99.908% when pulling 5db2617 on lukpueh:add-gpg into d6227b4 on secure-systems-lab:master.

[coveralls]

Coverage decreased (-0.05%) to 99.954% when pulling 9906acd on lukpueh:add-gpg into ae8517d on secure-systems-lab:master.

[lukpueh]

I just rebased onto master, force pushed and lost the merge commits that contained messages about how I used git filter-branch with --subdirectory-filter and --index-filter to preserve the git history of the gpg subpackage (and the process module).

For documentation purposes here is what I did to pull in the gpg subpackage together with its history:

# Grab in-toto repo and create a new branch at tip of development
  git clone [email protected]:in-toto/in-toto.git
  cd in-toto
  git checkout -b in-toto-gpg

  # Make "in_toto/gpg" subdir the root of the repo
  git filter-branch --prune-empty --subdirectory-filter in_toto/gpg HEAD

  # Grab securesystemslib repo and fetch the unrelated in-toto branch that
  # only has the contents of in-toto's gpg subdirectory
  cd ..
  git clone [email protected]:secure-systems-lab/securesystemslib.git
  cd securesystemslib
  git remote add in-toto ../in-toto
  git fetch in-toto
  git checkout in-toto-gpg

  # Move all gpg files back into a corresponding securesystemslib/gpg subdir
  # NOTE: sed does not understand \t so we use a literal tab (Ctrl-v + TAB)
  git filter-branch --index-filter \
    'git ls-files -s | sed "s/$(printf \\t)/&\securesystemslib\/gpg\//" |
      GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
        git update-index --index-info &&
     mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD

  # Merge the unrealted gpg branch into a new working branch based off of
  # securesystemslib's master
  git checkout master
  git checkout -b add-gpg
  git merge in-toto-gpg --allow-unrelated-histories

[lukpueh]

Okay, I just cleaned up the history here. This is ready for reviewing. As mentioned above in the PR description, only lukpueh/securesystemslib@47a9d9a...lukpueh:add-gpg is new code. The rest of the history was transferred from the in-toto repository.

@rennergade, do you want to give it a try?

[rennergade]

This looks good. This is my first code review so might not have the sharpest eye.

One general comment is that it seems the imports at the beginning of this file break the conventions outlined here. I'm not sure if this is purposely ignored.

[rennergade]

As an outsider, it's a little unclear to me how is_hashed is verified. I think a comment is warranted on why the index is being compared to the length.

[lukpueh]

Let me know if 9906acd works for you.

[lukpueh]

Thanks for the quick review, @rennergade!

We've been a little bit lenient regarding import-convention on the in-toto repo (where this code was written originally). There is a corresponding issue over there, but we labeled it low-prio (see in-toto/in-toto#145).

[rennergade]

Looks great, much clearer.