Schema changes for tags and using Akka Serialization for payloads #467

chbatey · 2020-12-11T10:10:38Z

Raising so can keep track of progress. Plenty still to do

patriknw

looking good

core/src/test/resources/schema/mysql/mysql-schema-v5.sql

core/src/test/resources/schema/postgres/postgres-schema-v5.sql

octonato

@chbatey, I did a first pass. I will do a more detailed one later.
Left a few comments here and there.

I think we should rebase this with #465 (if I get it green). Some important pieces have moved.

We should think how we want to test the old DAO, maybe we should have one job (Scala 2.13, JDK11 and old DAO).

One more thing, some headers are missing, this will show up in the build. Need to run a sbt headersCreateAll.

octonato · 2020-12-15T13:24:39Z

core/src/main/resources/reference.conf

@@ -259,7 +264,7 @@ jdbc-snapshot-store {
  #   to the same value for these other journals.
  use-shared-db = null

-  dao = "akka.persistence.jdbc.snapshot.dao.ByteArraySnapshotDao"
+  dao = "akka.persistence.jdbc.snapshot.dao.AkkaSerializerSnapshotDao"


I think we can call it DefaultSnapshotDao and DefaultReadJournalDao and we don't need to expose the impl detail that it's using Akka serialization.

This is also meant to be the default DAO, so ok to have a short name.

may I insist on changing the namesto DefaultSnapshotDao and DefaultReadJournalDao? 😉

core/src/main/scala/akka/persistence/jdbc/config/AkkaPersistenceConfig.scala

core/src/main/scala/akka/persistence/jdbc/journal/dao/AkkaSerializerJournalDao.scala

core/src/main/scala/akka/persistence/jdbc/journal/dao/legacy/ByteArrayJournalDao.scala

core/src/main/scala/akka/persistence/jdbc/journal/dao/AkkaSerializerJournalDao.scala

core/src/main/scala/akka/persistence/jdbc/package.scala

core/src/main/resources/reference.conf

patriknw · 2021-01-04T08:42:58Z

What is remaining here? Anything that needs additional review?

chbatey · 2021-01-04T08:48:57Z

Ready for a final review. Nothing specific to look at.

…

On Mon, Jan 4, 2021 at 8:43 AM Patrik Nordwall ***@***.***> wrote: What is remaining here? Anything that needs additional review? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#467 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOHYG5IXZSCQVWWB2TCPELSYF5SBANCNFSM4UWM4DWA> .

octonato

Looking really good.

I think we are almost there. I left a few comments and suggestions here and there.

Some things needs to be tackled before merging tough.

.travis.yml

octonato · 2021-01-05T15:17:51Z

core/src/main/resources/reference.conf

@@ -259,7 +264,7 @@ jdbc-snapshot-store {
  #   to the same value for these other journals.
  use-shared-db = null

-  dao = "akka.persistence.jdbc.snapshot.dao.ByteArraySnapshotDao"
+  dao = "akka.persistence.jdbc.snapshot.dao.AkkaSerializerSnapshotDao"


may I insist on changing the namesto DefaultSnapshotDao and DefaultReadJournalDao? 😉

core/src/main/resources/reference.conf

core/src/main/resources/schema/mysql/mysql-create-schema-v5.sql

core/src/main/scala/akka/persistence/jdbc/journal/dao/JournalQueries.scala

core/src/main/scala/akka/persistence/jdbc/testkit/internal/SchemaType.scala

core/src/main/scala/akka/persistence/jdbc/testkit/internal/SchemaUtilsImpl.scala

core/src/test/resources/oracle-schema-overrides.conf

core/src/test/scala/akka/persistence/jdbc/query/JournalDaoStreamMessagesMemoryTest.scala

core/src/test/scala/akka/persistence/jdbc/query/JournalSequenceActorTest.scala

chbatey · 2021-01-08T08:44:47Z

may I insist on changing the namesto DefaultSnapshotDao and DefaultReadJournalDao

If you must but i think it is inconsistent with the name of the other impl which does include how it serialises in the name

core/src/main/scala/akka/persistence/jdbc/testkit/internal/SchemaUtilsImpl.scala

docs/src/main/paradox/index.md

ennru

LGTM!

chbatey added 6 commits December 10, 2020 10:08

Journal spec passing for postgres, not writing tags yet

9ec0cf9

Working for postgres

dff2338

New schema for snapshots, works for postgres

b0f4299

Metadata suport for events

5fb1e68

Metadata support for snapshots

308148a

Mysql working

01f1a50

patriknw reviewed Dec 11, 2020

View reviewed changes

chbatey added 3 commits December 11, 2020 13:19

Review feedback

9562cc0

Sql server passing

e5acba8

oracle not working

a91ab63

octonato reviewed Dec 15, 2020

View reviewed changes

chbatey added 7 commits December 15, 2020 15:35

Finnnnnaalllyy oracle is working

835761b

H2 schema working

bf22e2e

Events by tag query working

158e1b9

Really work for H2 this time

20c963f

Remove mima for now

7334da1

headers

f42164e

Add test for old dao

2525ec8

chbatey force-pushed the wip-chbatey-new-schema branch from 8ce0f4c to 2525ec8 Compare December 16, 2020 10:19

chbatey added 8 commits December 16, 2020 10:57

Create schema based on dao in use

8230d6b

Doh

8174d36

Fix schemas

92ef3f5

Fix DB cleanup

687dae2

More oracle fixes

729ddb6

Disable tests for newDao that insert directly

fbb21a6

More oracle fixes

4d82660

Fix oracle event adapter test for old dao

2ae32b2

chbatey force-pushed the wip-chbatey-new-schema branch from 2041ddd to 2ae32b2 Compare December 17, 2020 08:16

chbatey added 2 commits December 17, 2020 08:54

Fix oracle event adapter test for old dao

7a9d26f

Fix sql server cleanup

96f9677

Minor oracle schema changes

3f317c5

ennru reviewed Dec 22, 2020

View reviewed changes

core/src/main/resources/reference.conf Outdated Show resolved Hide resolved

core/src/main/resources/reference.conf Outdated Show resolved Hide resolved

chbatey added 3 commits December 22, 2020 14:10

Fix oracle drops

5d33206

Doh broke oracle

822d6ec

Fix typo

2baf0f0

chbatey marked this pull request as ready for review December 23, 2020 09:16

chbatey changed the title ~~[Wip] Schema changes for tags and using Akka Serialization for payloads~~ Schema changes for tags and using Akka Serialization for payloads Dec 23, 2020

Add actor system name to test to see which logs are for what test

17566ec

octonato approved these changes Jan 5, 2021

View reviewed changes

chbatey added 4 commits January 8, 2021 09:07

Run old dao tests one db at a time

6208c0b

Rename DAOs to default

3c8599c

Throw/fail future if snapshot deserialization fails

e2fc20c

Last bits of review feedback

b5a6849

ennru reviewed Jan 8, 2021

View reviewed changes

core/src/main/scala/akka/persistence/jdbc/testkit/internal/SchemaUtilsImpl.scala Outdated Show resolved Hide resolved

docs/src/main/paradox/index.md Outdated Show resolved Hide resolved

chbatey added 3 commits January 8, 2021 09:54

Doc the new schema not the legacy

4d85fb6

Fix travis file

37cb27d

Fix newDao check

3de9283

ennru approved these changes Jan 8, 2021

View reviewed changes

ennru merged commit 0d2f1ee into akka:master Jan 8, 2021

This was referenced Jan 11, 2021

New Snapshot Store Schema #316

Closed

New Journal Store Schema #315

Closed

ennru mentioned this pull request Jan 11, 2021

New schema and migration for 5.0.0 #318

Closed

octonato mentioned this pull request Jan 25, 2021

eventsByTag query returns incorrect results for tags that share a prefix #168

Closed

ignasi35 mentioned this pull request Jul 23, 2021

Change the default for akka-persistence-jdbc.logicalDeletion.enable #557

Open

ismaelhamed mentioned this pull request Oct 5, 2021

Akka.Persistence.Sql.Common: add separate tags table for indexing individual tags akkadotnet/akka.net#5296

Closed

ismaelhamed mentioned this pull request Jun 18, 2022

Tag table with backwards compatibility akkadotnet/Akka.Persistence.Sql#68

Merged

12 tasks

wjohnsonbynder mentioned this pull request Feb 8, 2024

Slow JDBC query for eventsByTag apache/pekko-persistence-jdbc#105

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Schema changes for tags and using Akka Serialization for payloads #467

Schema changes for tags and using Akka Serialization for payloads #467

chbatey commented Dec 11, 2020 •

edited

Loading

patriknw left a comment

octonato left a comment

octonato Dec 15, 2020

octonato Jan 5, 2021

patriknw commented Jan 4, 2021

chbatey commented Jan 4, 2021 via email

octonato left a comment

octonato Jan 5, 2021

chbatey commented Jan 8, 2021

ennru left a comment

Schema changes for tags and using Akka Serialization for payloads #467

Schema changes for tags and using Akka Serialization for payloads #467

Conversation

chbatey commented Dec 11, 2020 • edited Loading

patriknw left a comment

Choose a reason for hiding this comment

octonato left a comment

Choose a reason for hiding this comment

octonato Dec 15, 2020

Choose a reason for hiding this comment

octonato Jan 5, 2021

Choose a reason for hiding this comment

patriknw commented Jan 4, 2021

chbatey commented Jan 4, 2021 via email

octonato left a comment

Choose a reason for hiding this comment

octonato Jan 5, 2021

Choose a reason for hiding this comment

chbatey commented Jan 8, 2021

ennru left a comment

Choose a reason for hiding this comment

chbatey commented Dec 11, 2020 •

edited

Loading