-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Way to 0.1 #1
Comments
Dealing with
|
@ilslv regarding serialization/deserialization I'd like to temporary keep that story aside from this project. Ideally, we shouldn't dictate this to library users at all. So implementing serialize/deserialize is totally their responsibility, not ours. The question which needs discussion and investigation here is not about serialization/deserialization, but rather about Ideally, we don't want new events to wear the burden of its predecessors, and we want clear ways to avoid outdated versions and work only with new ones. For example, we had this initially: #[derive(event::Versioned)]
#[event(name = "email.added", version = 1)]
struct EmailAddedV1 {
email: String,
} And later we evolve to something like that: #[derive(event::Versioned)]
#[event(name = "email.added", version = 2)]
struct EmailAddedV2 {
email: String,
by: UserId,
} And we have dilemma here:
Another question to investigate is how to better keep outdated events (modules layout, etc). |
Evolving schema1. Extending an already existing eventMost of the time in this case we'll just add fields to some event. There are 3 different ways of dealing with this situation:
/// Old event
#[derive(event::Versioned)]
#[event(name = "email.added", version = 1)]
struct EmailAddedV1 {
email: String,
}
// 1. Creating `From` implementation
#[derive(event::Versioned)]
#[event(name = "email.added", version = 2)]
struct EmailAddedV2 {
email: String,
confirmed_by: Option<UserId>,
}
impl From<EmailAddedV1> for EmailAddedV2 {
// ...
}
impl Sourced<EmailAddedV2> for S {
// ...
}
// How it may look in the future
#[derive(event::Versioned)]
#[event(name = "email.added", version = 2)]
struct EmailAddedV10 {
email: String,
confirmed_by: Option<UserId>,
a: Option<A>,
lot: Option<Lot>,
of: Option<Of>,
optional: Option<Optional>,
fields: Option<Fields>,
}
impl From<EmailAddedV1> for EmailAddedV10 {
// ...
}
// ...
impl From<EmailAddedV9> for EmailAddedV10 {
// ...
}
impl Sourced<EmailAddedV10> for S {
// ...
}
// 2. Creating `From` implementation
#[derive(event::Versioned)]
#[event(name = "email.added", version = 2)]
struct EmailAddedV2 {
email: String,
confirmed_by: UserId,
}
impl Sourced<EmailAddedV1> for S {
// ...
}
impl Sourced<EmailAddedV2> for S {
// ...
}
// How it may look in the future
impl Sourced<EmailAddedV4> for S {
// ...
}
// ...
impl Sourced<EmailAddedV10> for S {
// ...
}
// 3. Uniting events of different versions in enums
#[derive(event::Versioned)]
#[event(name = "email.added", version = 2)]
struct EmailAddedV2 {
email: String,
confirmed_by: UserId,
}
enum EmailAdded {
V1(EmailAddedV1)
V2(EmailAddedV2)
}
impl Sourced<EmailAdded> for S {
// ...
}
// How it may look in the future
#[derive(event::Versioned)]
#[event(name = "email.added", version = 9)]
struct EmailAddedLegacy {
email: String,
confirmed_by: Option<UserId>,
a: Option<A>,
lot: Option<Lot>,
of: Option<Of>,
optional: Option<Optional>,
fields: Option<Fields>,
}
#[derive(event::Versioned)]
#[event(name = "email.added", version = 10)]
struct EmailAddedV10 {
email: String,
confirmed_by: Option<UserId>,
much: Much,
stricter: Stricter,
definition: Definition,
}
enum EmailAdded {
Legacy(EmailAddedLegacy), // Converted from versions 1-9
V10(EmailAddedV10),
}
impl Sourced<EmailAdded> for S {
// ...
} 2. Renaming/removing event's fields
#[derive(event::Versioned)]
#[event(name = "email.added", version = 1)] // Version didn't change
struct EmailAddedV2 {
#[event(alias(value))]
email: String,
}
#[derive(event::Versioned)]
#[event(name = "email.added", version = 2)]
struct EmailAddedV2 {
#[event(alias(value, version = 1))]
email: String,
}
// May be expanded to different structs or remain single with version validation on deserialization Both 1 and 2 are requiring to enforce our own deserialization onto developer. I don't consider that as much of a problem, as
#[derive(event::Versioned)]
#[event(name = "email.added", version = 1)]
struct EmailAddedV1 {
value: String,
}
#[derive(event::Versioned)]
#[event(name = "email.added", version = 2)]
struct EmailAddedV2 {
email: String,
}
impl From<EmailAddedV1> for EmailAddedV2 {
// ...
}
/// May be combined with `3. Uniting events of different versions in enums` from previous step
I lean more to this option, as renaming fields should be quite infrequent usecase 3. Ignore entire event
4. Split large event
5. Transforming events based on some ContextWe can't gurantee that it would be possible to deterministically transform old event into a new one (althought it should be the last resort), so ProposalFor solving first 2 problems I propose combination of 1.3 and 2.3. This should cover us for most use-cases. // Declarations of events 1-8 with `Deserialize` and `From` impls for `EmailAddedLegacy`
#[derive(event::Versioned, Deserialize)]
#[event(name = "email.added", version = 9)]
struct EmailAddedLegacy {
email: String,
confirmed_by: Option<UserId>,
a: Option<A>,
lot: Option<Lot>,
of: Option<Of>,
optional: Option<Optional>,
fields: Option<Fields>,
}
#[derive(event::Versioned, Deserialize)]
#[event(name = "email.added", version = 10)]
struct EmailAddedV10 {
email: String,
confirmed_by: Option<UserId>,
much: Much,
stricter: Stricter,
definition: Definition,
}
#[derive(Event, Deserialize)]
enum EmailAdded {
Legacy(EmailAddedLegacy), // Converted from versions 1-9
V10(EmailAddedV10),
}
impl Sourced<EmailAdded> for S {
// ...
} Regarding problems 3-5 it looks like we sould add a new abstraction layer between event storage and Unresolved questionsShould we consider blue-green deployment where some instances of the same service are producing old events, when other instances already were upgraded? ack @tyranron |
Discussed:
Backwards-compatibility is preserved (when old versions of the events are stored for a short amount of time), while forward-compatibility is not, which resolves in
Sounds like the way to go |
First draft of EventAdapterBase traittrait EventTransformer<Event> {
type Context: ?Sized;
type Error;
type TransformedEvent;
type TransformedEventStream<'ctx>: Stream<Item = Result<
Self::TransformedEvent,
Self::Error,
> + 'ctx;
fn transform(
event: Event,
context: &mut Self::Context,
) -> Self::TransformedEventStream<'_>;
}
Design decisions
AlternativesReplace Convenience traitstrait EventTransformStrategy<Event> {
type Strategy;
} To avoid implementing everything by hand we would provide some convenience impl EventTransformStrategy<SkippedEvent> for Adapter {
type Strategy = strategy::Skip;
}
impl EventTransformStrategy<EmailConfirmed> for Adapter {
type Strategy = strategy::AsIs;
}
impl EventTransformStrategy<EmailAdded> for Adapter {
type Strategy = strategy::Into<EmailAddedOrConfirmed>;
}
impl EventTransformStrategy<EmailAddedAndConfirmed> for Adapter {
type Strategy = strategy::Split<EmailAddedOrConfirmed, 2>;
}
impl From<EmailAddedAndConfirmed> for [EmailAddedOrConfirmed; 2] {
fn from(ev: EmailAddedAndConfirmed) -> Self {
[
EmailAdded { email: ev.email }.into(),
EmailConfirmed {
confirmed_by: ev.confirmed_by,
}
.into(),
]
}
}
These are just examples and we can provide many more Besides that, we didn't loose ability to implement impl EventTransformer<Custom> for Adapter {
type Context = dyn Any;
type Error = Infallible;
type TransformedEvent = EmailAddedOrConfirmed;
type TransformedEventStream<'ctx> = stream::Empty<Result<EmailAddedOrConfirmed, Infallible>>;
fn transform(
_: Custom,
_: &mut Self::Context,
) -> Self::TransformedEventStream<'_> {
stream::empty()
}
} That impl basically is the same as trait EventAdapter<Events> {
type Context: ?Sized;
type Error;
type TransformedEvents;
type TransformedEventsStream<'ctx>: Stream<Item = Result<Self::TransformedEvents, Self::Error>>
+ 'ctx;
fn transform_all(
events: Events,
context: &mut Self::Context,
) -> Self::TransformedEventsStream<'_>;
}
impl<Adapter, Events> EventAdapter<Events> for Adapter
where
Events: Stream + 'static,
Adapter: EventTransformer<Events::Item> + 'static,
Adapter::Context: 'static,
{
type Context = Adapter::Context;
type Error = Adapter::Error;
type TransformedEvents = Adapter::TransformedEvent;
type TransformedEventsStream<'ctx> = AdapterStream<'ctx, Adapter, Events>;
fn transform_all(
events: Events,
context: &mut Self::Context,
) -> Self::TransformedEventsStream<'_> {
AdapterStream::new(events, context)
}
} This trait comes with a blanket impl for any compatible type, implementing Whole implementation flow// Declare all possible input events
#[derive(Debug)]
struct SkippedEvent;
#[derive(Debug)]
struct EmailAddedAndConfirmed {
email: String,
confirmed_by: String,
}
#[derive(Debug)]
struct EmailAdded {
email: String,
}
#[derive(Debug)]
struct EmailConfirmed {
confirmed_by: String,
}
// Unite them in a enum, deriving `EventTransformer`
#[derive(Debug, From, EventTransformer)]
#[event(transform(into = EmailAddedOrConfirmed, context = dyn Any))]
enum InputEmailEvents {
Skipped(SkippedEvent),
AddedAndConfirmed(EmailAddedAndConfirmed),
Added(EmailAdded),
Confirmed(EmailConfirmed),
}
// Declare enum of output events
#[derive(Debug, From)]
enum EmailAddedOrConfirmed {
Added(EmailAdded),
Confirmed(EmailConfirmed),
}
// Implement transformations
struct Adapter;
impl EventTransformStrategy<EmailAdded> for Adapter {
type Strategy = strategy::AsIs;
}
impl EventTransformStrategy<EmailConfirmed> for Adapter {
type Strategy = strategy::Into<EmailAddedOrConfirmed>;
}
impl EventTransformStrategy<EmailAddedAndConfirmed> for Adapter {
type Strategy = strategy::Split<EmailAddedOrConfirmed, 2>;
}
impl From<EmailAddedAndConfirmed> for [EmailAddedOrConfirmed; 2] {
fn from(ev: EmailAddedAndConfirmed) -> Self {
[
EmailAdded { email: ev.email }.into(),
EmailConfirmed {
confirmed_by: ev.confirmed_by,
}
.into(),
]
}
}
impl EventTransformStrategy<SkippedEvent> for Adapter {
type Strategy = strategy::Skip;
}
// Test Adapter
#[tokio::main]
async fn main() {
let mut ctx = 1_usize; // Can be any type
let events = stream::iter::<[InputEmailEvents; 4]>([
EmailConfirmed {
confirmed_by: "1".to_string(),
}
.into(),
EmailAdded {
email: "2".to_string(),
}
.into(),
EmailAddedAndConfirmed {
email: "3".to_string(),
confirmed_by: "3".to_string(),
}
.into(),
SkippedEvent.into(),
]);
let collect = Adapter::transform_all(events, &mut ctx)
.collect::<Vec<_>>()
.await;
println!("context: {}\nevents:{:?}", ctx, collect);
// context: 1,
// events: [
// Ok(Confirmed(EmailConfirmed { confirmed_by: "1" })),
// Ok(Added(EmailAdded { email: "2" })),
// Ok(Added(EmailAdded { email: "3" })),
// Ok(Confirmed(EmailConfirmed { confirmed_by: "3" }))
// ]
} DownsidesTo implement ack @tyranron |
|
Agreed, especially that in practice
Very good point, entirely missed it
It's just PoC, of course we should provide some mechanism, that will allow to vary number of emitted elements based on content of input event, while array approach doesn't |
- bootstrap project structure Additionally: - bootstrap CI pipeline - bootstrap Makefile Co-authored-by: tyranron <[email protected]>
…to single Rust type (#3, #1) Additionally: - support event::Initial in derive macros Co-authored-by: tyranron <[email protected]>
- add event::Sourcing trait for more handy dynamic dispatch usage Co-authored-by: tyranron <[email protected]>
@tyranron regarding our discussion how |
Project layout
arcana-core
(core/
dir): contains core abstractionsarcana-codegen-impl
(codegen/impl/
dir): contains codegen implementationsarcana-codegen
(codegen/
dir): proc-macro shim crate forarcana-codegen-impl
arcana
(project root): umbrella crate uniting all others behind feature-gatesRoadmap
Makefile
The text was updated successfully, but these errors were encountered: