JOSE support

[whyoleg]

Introduce standalone JOSE support module with JWT/JWK/etc.
Support for JWK encoding/decoding can also be used afterwards in providers other than WebCrypto.

Full list of RFCs:

• JWS: https://datatracker.ietf.org/doc/html/rfc7515
• JWE: https://datatracker.ietf.org/doc/html/rfc7516
• JWK: https://datatracker.ietf.org/doc/html/rfc7517
• JWA: https://datatracker.ietf.org/doc/html/rfc7518
• JWT: https://datatracker.ietf.org/doc/html/rfc7519
• JOSE examples: https://datatracker.ietf.org/doc/html/rfc7520

[malhobayyeb]

Up

[waltkb]

Hi, is there anything we (walt.id) could maybe help implement here?

[whyoleg]

Hey, @waltkb, thank you for the interest! That would be really nice!
I've created a branch jose (from main), and we could start creating PR's there as a start.
What I wanted from the initial version is to implement JWT and JWK support. I have no time to investigate the API which would fit Kotlin, so overall, it is up to you to start, and I will review PRs as we go.
To begin, let us add jose module under cryptography-serialization folder. You can also take a look on setup of asn1 module as a base for it — all targets + kotlinx-serialization-json. No other dependencies, at least for now.
The main idea would be to be simple enough and use kotlinx-serialization-json to handle serialisation (obviously).
Also, we will need Base64 - I would propose to just use the one which is already available in kotlin-stdlib. It is experimental, but I believe it should be fine.
If you have any other questions, feel free to ask here or anywhere else linked on GitHub!

When we have all the base blocks from the serialisation part, then we can start thinking about how to integrate it with signing part of core module to have the best experience.

Thank you once more, I'm looking forward to it!

[aschulz90]

I am currently in the progress of implementing a JWT library for Kotlin Multiplatform supporting all platforms this library also supports and have signing and verification done very easily so far.
Next step is JWKS support, but is is currently blocked by missing support for decoding public keys using JWK format.

The current progress is avaiable here: https://github.com/Appstractive/jwt-kt
The JWK handling is implemented here.

Maybe the repository will be useful to you by using it as a playground to test the JWK implementation.
There is a test in the jwt-jwks module, that should pass, once JWK decoding works for all platforms (pending interface changes).

[babisRoutis]

Hey, @waltkb, thank you for the interest! That would be really nice! I've created a branch jose (from main), and we could start creating PR's there as a start. What I wanted from the initial version is to implement JWT and JWK support. I have no time to investigate the API which would fit Kotlin, so overall, it is up to you to start, and I will review PRs as we go. To begin, let us add jose module under cryptography-serialization folder. You can also take a look on setup of asn1 module as a base for it — all targets + kotlinx-serialization-json. No other dependencies, at least for now. The main idea would be to be simple enough and use kotlinx-serialization-json to handle serialisation (obviously). Also, we will need Base64 - I would propose to just use the one which is already available in kotlin-stdlib. It is experimental, but I believe it should be fine. If you have any other questions, feel free to ask here or anywhere else linked on GitHub!

When we have all the base blocks from the serialisation part, then we can start thinking about how to integrate it with signing part of core module to have the best experience.

Thank you once more, I'm looking forward to it!

Dear @whyoleg

The branch that you mentioned (jose) is not present. Was this postponed for a future release?
You proposed to add a jose module under cryptography-serialization folder.
To my understanding similar modules like der / pem provide some functions to the rest of the modules. Thus, they don't depend on other (project) modules.

Can this be the case of jose?
For instance, I was thinking that to support JWK, the jose module should provide a (JSON Serializer) for ECKeys, thus it should depend on the core. An alternative could be to define some serializable classes (for instance ECPubKeyJWK) in jose a let somebody else to use them.

Can you please elaborate?

[whyoleg]

Hey @babisRoutis !

The branch that you mentioned (jose) is not present.

Yeah, looks like I deleted it while preparing the release. Anyway, it was just the same as main at that point to initiate discussion/implementation, as I would think, that probably it will take not a single PR to complete. I could create it once more, if you do want to contribute :)

Was this postponed for a future release?

It was planned for 0.5.0 (next "major") release for a long time, as I had other things to do.
I've just released 0.4.0 and now I need to switch to other projects for some time, so I'm not planning to work on it right.
But this is still on my radar as the first feature for the next major release.

Can this be the case of Jose?

Yeah, sure, I think so! At least initially we should have all data models for serialization (similar to how ASN.1/DER). Only after that we could start thinking about integration.
The story with JWK is rather easy, as in this case cryptography-serialization-jose will be just an implementation detail of providers. JWK is already supported as a key format in core API.

The story with JWT is harder, as it will probably need more tight integration with core APIs like signatures. For this, I think, that may be we will be able to create an additional module, a.k.a. cryptography-jwk where there will be high-level API to work with JWT and cryptography which under the hood will use cryptography-serialization-jose.

Do you get the idea?
It's not final, but at least this is what I have in mind.
If you are willing to help with contribution on this or do have some specific use-cases apart from JWK support, I would like to hear them and connect here/slack/call to discuss if we can cooperate on this.

P.S. The library size, complexity and amount of moving parts is only increasing and so working on it alone becomes harder and harder so I would really like if the work could be divided :)

[whyoleg]

Hey @aschulz90 !
Thanks for your work and links! This is really useful.
BTW, may be you do want to somehow cooperate on JWK support in cryptography-kotlin based on the plan in the message above?

[whyoleg]

On additional note, last time I thought about how JWK is implemented currently in the library, I was not very sure if it's a good idea to have JWK embedded into encode/decode APIs.
The reason for its existence is because WebCrypto supports JWK out-of-the-box. Google's tink also supports JWK, so I thought it looks natural.
But now, when I was tried to add some of the algorithms, it filled a bit unnatural, as DER/PEM encoding in some cases needs less parameters, then needed for JWK.
Also, as JWK specify algorithm inside key, it would be nice to have an ability to somehow create signer/verifier based on just key without a requirement to specify expected algorithm (or digest) first.
I don't yet know if it possible to do this in a nice way. That's why having cryptography-serialization-jose independent from core would be a nice thing, as we could easier experiment with different ways of encoding/decoding of keys. May be in future, JWK will even be removed from encode/decode and used in another way.

I don't know yet, just leaving my thoughts here.
P.S. also, there is COSE, for which I don't have plans yet, but it would be nice to keep in mind on how it should be integrated on par with JOSE.

[babisRoutis]

But now, when I was tried to add some of the algorithms, it filled a bit unnatural, as DER/PEM encoding in some cases needs less parameters, then needed for JWK. Also, as JWK specify algorithm inside key, it would be nice to have an ability to somehow create signer/verifier based on just key without a requirement to specify expected algorithm (or digest) first. I don't yet know if it possible to do this in a nice way. That's why having cryptography-serialization-jose independent from core would be a nice thing, as we could easier experiment with different ways of encoding/decoding of keys. May be in future, JWK will even be removed from encode/decode and used in another way.

This is the reason that triggered my question.

To my understanding, having a no-deps JWK serialization module, means that core would need to adapt it, yet JOSE(and COSE) come with their own quirks that eventually will "pollute" core.

Perhaps, jose could be a top-level module, or even a new lib, that just consumes cryptography-kotlin (similar to @aschulz90 work).

[aschulz90]

Thanks for your work and links! This is really useful.
BTW, may be you do want to somehow cooperate on JWK support in cryptography-kotlin based on the plan in the message above?

I have started to look into it some time ago, but I am not that deep into cryptography and was kinda ovewhelmed by the RFC and its terminology. From what I gathered, the model could look like this:

import dev.whyoleg.cryptography.CryptographyAlgorithmId
import dev.whyoleg.cryptography.algorithms.digest.Digest
import dev.whyoleg.cryptography.algorithms.digest.SHA256
import dev.whyoleg.cryptography.algorithms.digest.SHA384
import dev.whyoleg.cryptography.algorithms.digest.SHA512
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.JsonClassDiscriminator


enum class Algorithm {
  HS256,
  HS384,
  HS512,
  RS256,
  RS384,
  RS512,
  PS256,
  PS384,
  PS512,
  EC256,
  EC384,
  EC512,
}

val Algorithm.digest: CryptographyAlgorithmId<Digest>
  get() =
      when (this) {
        Algorithm.EC256,
        Algorithm.PS256,
        Algorithm.RS256,
        Algorithm.HS256 -> SHA256

        Algorithm.EC384,
        Algorithm.RS384,
        Algorithm.PS384,
        Algorithm.HS384 -> SHA384

        Algorithm.EC512,
        Algorithm.HS512,
        Algorithm.RS512,
        Algorithm.PS512 -> SHA512
      }

enum class Curve {
  @SerialName("P-256") P256,
  @SerialName("P-384") P384,
  @SerialName("P-521") P521,
}

@OptIn(ExperimentalSerializationApi::class)
@Serializable
@JsonClassDiscriminator("kty")
sealed interface JSONWebKey {
  val alg: Algorithm?
  val kid: String?
}

@Serializable
@SerialName("RSA")
data class JSONWebKeyRSA(
    override val alg: Algorithm? = null,
    override val kid: String,
    val use: String,
    val n: String,
    val e: String,
    val d: String? = null,
    val p: String? = null,
    val q: String? = null,
    val dp: String? = null,
    val dq: String? = null,
    val qi: String? = null,
) : JSONWebKey

@Serializable
@SerialName("EC")
data class JSONWebKeyEC(
  override val alg: Algorithm? = null,
  override val kid: String,
  val crv: Curve,
  val x: String,
  val y: String,
  val d: String,
) : JSONWebKey

@Serializable
@SerialName("oct")
data class JSONWebKeyHMAC(
    override val alg: Algorithm? = null,
    override val kid: String,
    val k: String,
) : JSONWebKey

@Serializable
data class JSONWebKeySet(
    val keys: List<JSONWebKey>,
)

But I am not sure about the parameters, or if they even are all required to genrate the public key from.

it would be nice to have an ability to somehow create signer/verifier based on just key without a requirement to specify expected algorithm (or digest) first.

This is how I currently generate the verifier based on the above model class: JSONWebKeyExt.kt

As for the actual generation of public keys: at least on JVM it seems to be pretty simple, e.g. like this: jwks-rsa-java

[whyoleg]

Okay, I will try to expand on the ideas which I have right now in mind and may be this will help to understand what is the direction which I see for JOSE support in cryptography-kotlin (CK for short). This is not the final decision of course, more of a starting point.

I see the solution which consists of three parts:

1. Data models for JOSE (a.k.a cryptography-serialization-jose)

Overall, all of the JOSE RFCs are mostly about how to represent cryptography related data (keys, algorithms, signatures, tokens) via JSON or text (in case of JWT). So as a start point, we need to have an ability to encode/decode those models between textual representation and code.
For this we will need to create models, probably mostly the same as shown by @aschulz90 in the message above. May be with slight naming changes.
Those should really depend only on kotlinx-serialization and not on core and so have no access to cryptography functionality (e.g signing/encryption) at all.
Probably in the end it could be just implementation detail, but also could be exposed to end users (TBD).

2. API in cryptography-core to support decoding keys without knowing the algorithm upfront

This API should provide something similar to JSONWebKeyExt.kt. Additionally, the same is API could be used for PEM/DER decoding, as PKCS8 (for private keys) and SPKI (for public keys) formats are also general and contains algorithm ID inside.
This API is not available at the moment and should be designed. It's also should be designed with implementation of Key and certificates management and storage (#19) and X509 certificates and PKCS12 #18 in mind, as all of those things are used for all the same things, but in different contexts.
That's why all those issues are grouped under 0.5.0 milestone - API design here should be consistent.

3. High-level APIs/libraries for JWT/JWKS/OAuth

Those APIs should be built on the points above. As an example of what could be there we could take a look on:

• jwt-kt by @aschulz90
• java-jwt or jwks-rsa-java from auth0
• or even things like Pac4j

Those should not necessarily be implemented inside CK, but could be maintained outside. While I do like to experiment with those inside of CK, I will probably have no time to implement all of the standards in the wild.

To conclude

It would be of course nice to have single library to work with cryptography in kotlin, but probably it will be never possible as the domain is rather big. So I do want for CK to provide most of the building blocks which are needed to implement all those high-level solutions outside. That's why I'm trying to look on those issues from a very high-level and long perspective. This is one of the reasons why releases are not that frequent :)

Of course, I'm open to other ideas and discussions.
Hope that I'm described my ideas understandable.

[aschulz90]

@whyoleg thanks for the summary!

Probably in the end it could be just implementation detail, but also could be exposed to end users (TBD).

In a backend context, it would be nice to have the model classes exposed.
Either for providing your own JWKS endpoint (if this library also supports public key -> JWKS conversion) or fetching the JWKS from another endpoint (but one could also simply pass the response body text/bytes without deserialization).

[babisRoutis]

@whyoleg thanks for the summary!

Probably in the end it could be just implementation detail, but also could be exposed to end users (TBD).

In a backend context, it would be nice to have the model classes exposed. Either for providing your own JWKS endpoint (if this library also supports public key -> JWKS conversion) or fetching the JWKS from another endpoint (but one could also simply pass the response body text/bytes without deserialization).

Totally agree.

Not only for back-end, though. For instance, in the context of EUDI Wallet there are several cases where working with JWKs is required on mobile side.

[whyoleg]

Yeah, I understand that. I was not clear on what I mean by implementation detail. Those models should be definitely exposed by cryptography-serialization-jose, but probably could be just an implementation details for points 2 and 3 - it depends.
So yes, those use-cases should be supported.