Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: TypeUrl trait + Any encoding support #858

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions prost-types/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,11 @@ use core::i64;
use core::str::FromStr;
use core::time;

use prost::alloc::borrow::ToOwned;
use prost::alloc::format;
use prost::alloc::vec::Vec;
use prost::{DecodeError, EncodeError, Message, TypeUrl};

pub use protobuf::*;

// The Protobuf `Duration` and `Timestamp` types can't delegate to the standard library equivalents
Expand All @@ -33,6 +38,38 @@ pub use protobuf::*;
const NANOS_PER_SECOND: i32 = 1_000_000_000;
const NANOS_MAX: i32 = NANOS_PER_SECOND - 1;

impl Any {
/// Serialize this message proto as [`Any`].
pub fn from_message<M>(msg: &M) -> Result<Self, EncodeError>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these might benefit from a shorter from_msg/to_msg naming, but I went with existing precedent which seems to spell out the whole word message.

where
M: TypeUrl,
{
let type_url = M::TYPE_URL.to_owned();
let mut value = Vec::new();
Message::encode(msg, &mut value)?;
Ok(Any { type_url, value })
}

/// Decode the given message type `M` from [`Any`], validating that it has
/// the expected [`TypeUrl`].
pub fn to_message<M>(&self) -> Result<M, DecodeError>
where
M: Default + Sized + TypeUrl,
{
if self.type_url == M::TYPE_URL {
Ok(M::decode(&*self.value)?)
} else {
let mut err = DecodeError::new(format!(
"expected type URL: \"{}\" (got: \"{}\")",
M::TYPE_URL,
&self.type_url
));
err.push("unexpected type URL", "type_url");
Err(err)
}
}
}

impl Duration {
/// Normalizes the duration to a canonical format.
///
Expand Down
2 changes: 2 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,15 @@ pub use bytes;

mod error;
mod message;
mod type_url;
mod types;

#[doc(hidden)]
pub mod encoding;

pub use crate::error::{DecodeError, EncodeError};
pub use crate::message::Message;
pub use crate::type_url::TypeUrl;

use bytes::{Buf, BufMut};

Expand Down
19 changes: 19 additions & 0 deletions src/type_url.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
//! Support for associating a type URL with a [`Message`].

use crate::Message;

/// Associate a type URL with the given [`Message]`.
pub trait TypeUrl: Message {
/// Type URL for this [`Message`]. They take the form:
///
/// ```text
/// /<package>.<TypeName>
/// ```
///
/// For example:
///
/// ```text
/// /foo.bar.baz.MyTypeName
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the slash is part of the type URL, the type name is foo.bar.baz.MyTypeName which then gets encoded into the Any message as type.googleapis.com/foo.bar.baz.MyTypeName

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I guess this particular encoding is an artifact of the type URLs I'm working with and not the standard encoding.

I'm still unclear though if all type URLs are expected to be prefixed with https://type.googleapis.com/ or just Google's own well-known types, which is what the example is for.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is difference between type url and full type name of a protobuf message.

For instance the full type name is not expected to have slashes.

The Type URL in a google.Protobuf.Any is expected to have at least one slash.

So if this Type URL mentioned here is supposed to end up in an any then by specification, it should contain at least one slash.

Copy link

@testinginprod testinginprod Jul 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this TYPE URL we are talking about in this PR is supposed to be used to compose the TYPE URL of an any then we shouldn't be talking about TYPE URL but we should be talking about FULLNAME of a protobuf message, they're different concepts.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the type url in Any should have a slash, but the type name that we attach to the trait should just be the full type name without a slash.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then this trait should not be called TypeURL  it should be called Fullname.

And the way we match a type URL is not the correct one:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use case is populating the type_url field of an Any message.

If you're saying the type URL can always be computed from the fullname by prepending a slash, that's fine. That's what the implementation I'm using is expecting.

/// ```
const TYPE_URL: &'static str;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just went with a &'static str for now, rather than trying to use a special type which validates the URL.

In my mind the goal is to get prost-build to spit these out eventually, in which case they should be well-formed. But maybe down the road it would be possible to add a URL type with a const fn parser.

}