You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For some use cases of a write ahead log, I'd like to keep an in-memory representation (which gets recovered) of the contents of the log. When a checkpoint is made, I can drop the check-pointed information from memory since it is now part of the checkpoint.
One difficulty, as far as I can tell, is that the checkpoint method is called in a separate thread -- this means the following could happen:
Commit some data A to the log
Initiate a checkpoint
Commit some data B to the log
Actually perform the checkpoint
The problem I have is that at that point, the contents of A and B are in the in-memory representaiton, while I only want to checkpoint (and potentially drop) A. For my use case, I'm fine keeping the in-memory representation behind a Mutex (there should be only one writer to the log), which also simplifies a lot of the logic.
There are a few things that could help me:
If the result of committing data indicated that a checkpoint would be taken. I think this would only help if there was a lock, but then I could detect that a checkpoint would be taken after A, and arrange to have a separate in-memory representation started for data added after that.
If there was a "pre-checkpoint" hook that was called before other commits were allowed. Again, this would allow me to split the in-memory representation for B.
The text was updated successfully, but these errors were encountered:
Thank you for the idea for the additional use case for this crate. I think the easiest thing to offer would be your second suggestion.
If you need this quickly, I would love a pull request. Otherwise I will eventually be getting back onto this project, but right now it's not my primary focus.
For some use cases of a write ahead log, I'd like to keep an in-memory representation (which gets recovered) of the contents of the log. When a checkpoint is made, I can drop the check-pointed information from memory since it is now part of the checkpoint.
One difficulty, as far as I can tell, is that the checkpoint method is called in a separate thread -- this means the following could happen:
A
to the logB
to the logThe problem I have is that at that point, the contents of
A
andB
are in the in-memory representaiton, while I only want to checkpoint (and potentially drop)A
. For my use case, I'm fine keeping the in-memory representation behind a Mutex (there should be only one writer to the log), which also simplifies a lot of the logic.There are a few things that could help me:
A
, and arrange to have a separate in-memory representation started for data added after that.The text was updated successfully, but these errors were encountered: