Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Journal Store Schema #315

Closed
octonato opened this issue Jan 10, 2020 · 5 comments
Closed

New Journal Store Schema #315

octonato opened this issue Jan 10, 2020 · 5 comments
Assignees
Milestone

Comments

@octonato
Copy link
Member

octonato commented Jan 10, 2020

The new journal store table will have a few more fields:

  • ser_id (Int)
  • ser_manifest (String)
  • event_manifest (String)
  • event_payload (String)
  • writer_uuid (String)
  • timestamp column (see #28383). Useful for metrics about read-side processors lagging. The timestamp will be addes as Long and defaulting to 0.

We need to refactor the Journal DAO for that new format.

The current table is called journal. Instead of altering it, we will design a new table structure and call it journal_messages. New users can just start using it from version 4.0.0 onward. Existing users will need to run the migration procedure to copy their data from journal to journal_messages.

ByteArrayJournalSerializer should be moved to the migration module as we will need it for running migrations (reading old data from the journal table).

Next to the journal table, we need a tags table. There is a one-to-many relation between the two tables. One event can have zero or more tags.

The Journal DAO must take this into account and execute inserts on both tables atomically.

Events by tag queries need to be refactored to consider the new tags table.

@octonato octonato added this to the 4.0.0 milestone Jan 10, 2020
@patriknw
Copy link
Member

maybe call it journal_events instead of journal_messages? Akka Persistence naming of these things as messages is wrong in my opinion, but we don't have to replicate that problem to more places than necessary.

@octonato
Copy link
Member Author

Good. I confess that I was trying to align as much as possible. Cassandra calls it message while Couchbase is using journal_message in the singular.

@patriknw patriknw modified the milestones: 4.0.0, 5.0.0 Mar 30, 2020
@PerWiklander
Copy link

Aren't SQL tables typically named in the singular?

@chbatey
Copy link
Contributor

chbatey commented Dec 10, 2020

We also need to add columns for the metadata

@chbatey
Copy link
Contributor

chbatey commented Jan 11, 2021

#467

@ennru ennru closed this as completed Jan 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants