Using PyOpenSSL SSLContexts for verifying certificate chains without an SSL connection

[kislyuk]

Hi,

I use functionality that is unique to PyOpenSSL contexts (OpenSSL.SSL.Context) to verify certificate chains without having an SSL connection open. PyOpenSSL's trust store support is very limited (see notes in https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.set_default_verify_paths). Because of this, I've been falling back to certifi, but would prefer to make use of the functionality supported by truststore instead. Do you have any thoughts on the feasibility of this?

[sethmlarson]

Great question! I want to create a "_hazmat" API namespace that allows for users to use their own SSLContext implementation and sources of certificate chains with truststore. Might look pretty similar to the existing private APIs that change per OS.

cc @davisagli

[kislyuk]

HI @sethmlarson @davisagli! I'm back to inquire about the status of truststore and the future of using system CA trust roots for X509 based PKI verification.

A number of applications like various digital signature processing protocols use X509 PKI without SSL connections. It would be great if truststore could support a public API for certificate validation (with appropriate caveats/limitations since general purpose certificate validation is quite a complex task).

I've noticed there hasn't been as much activity in truststore recently - are there any blockers or architectural issues on the path to productization?

[sethmlarson]

@kislyuk There shouldn't be any blockers, I'm actually focusing a bit on this library currently because I'm trying to get pip to use it by default.

If you had time and knew what you were looking for approximately you could submit a PR otherwise I'll get to this eventually too :)

Thanks for the interest!

[kislyuk]

Thanks, that's fantastic news that truststore is under active development and aiming for integration with core Python tools.

I know what I'm looking for in theory (certificate validation using an API similar to https://github.com/wbond/certvalidator/blob/master/docs/api.md or at least https://www.pyopenssl.org/en/latest/api/crypto.html#OpenSSL.crypto.X509StoreContext) but I haven't tried sketching it out in practice based on what truststore provides.

I'll try putting together an API for this and get a PR going if I can figure it out.

[kislyuk]

Looking at this more intently now, here is the problem that I'm facing.

I'm looking to create an API like truststore.hazmat.verify(cert_chain_to_verify) which would verify the cert and its intermediates without an SSL context.

• On Windows, I can easily reuse the logic in truststore._windows._get_and_verify_cert_chain() and some of the setup logic in truststore._windows._verify_peercerts_impl(), perhaps modularizing some bits of the latter.
• On MacOS, I can also reuse relevant parts of truststore._macos._verify_peercerts_impl(), perhaps again modularizing it.
• On OpenSSL, I'm stuck. truststore._openssl._verify_peercerts_impl() is a no-op since we just rely on OpenSSL internals and treat them as a black box. In PyOpenSSL, I would use an X509StoreContext, but the stdlib ssl provides no equivalent and no bindings to the underlying OpenSSL functions (X509_STORE_new, X509_STORE_CTX_new etc).

Since the stdlib openssl is completely focused on SSL, it seems I have to separately get my own FFIs to the X509 Store functionality in OpenSSL with ctypes or something like that - but that kinda opens a whole OpenSSL management can of worms that I'm not sure we want to deal with here. I'll look at how pyOpenSSL and certvalidator/oscrypto do it to see if there is any easy/minimalistic way out, but it looks like a pretty fundamental problem.

[sethmlarson]

@kislyuk Yeah, this is the unfortunate part of the stdlib... Did you want to implement the API in two stages then? One for Windows+macOS and the second requiring more research potentially?

[kislyuk]

Yes, I think that's the practical path forward. My goal now is to get a PR going that basically does not implement this functionality on OpenSSL, only on Windows and MacOS. Supporting it on OpenSSL would be a second step that might require a separate optional dependency like cryptography.

[kislyuk]

@sethmlarson I added a sketch of what an API for this might look like in #102. I verified that it works as expected on MacOS and on OpenSSL. I wanted to run this by you before I add tests, documentation etc. to get your feedback first. What do you think?