Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for persistence #17

Closed
ezrasingh opened this issue Aug 16, 2024 · 0 comments · Fixed by #18
Closed

Add support for persistence #17

ezrasingh opened this issue Aug 16, 2024 · 0 comments · Fixed by #18
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@ezrasingh
Copy link
Owner

          The first step toward achieving HA might be to address fault tolerance by implementing persistence, as the current setup doesn’t support it—meaning a crash would result in total data loss. 

A possible solution could involve introducing state snapshots and an append-only log for operations. This would allow for some level of recovery during a failover, with configurable options for snapshot intervals.

My preference is for a memory-focused approach to persistence, where:

  1. A snapshot of the current state is taken.
  2. As operations are processed, they are logged.
  3. When the log reaches a certain size or memory threshold, a new snapshot is taken.
  4. The log file is either archived or simply discarded.
  5. The log is then flushed, and the process repeats.

Originally posted by @ezrasingh in #5 (comment)

@ezrasingh ezrasingh added the enhancement New feature or request label Aug 16, 2024
@ezrasingh ezrasingh self-assigned this Aug 16, 2024
@ezrasingh ezrasingh added this to the v1.0.0 milestone Aug 16, 2024
@ezrasingh ezrasingh linked a pull request Aug 23, 2024 that will close this issue
8 tasks
ezrasingh added a commit that referenced this issue Aug 23, 2024
* Added serde support for SpatialIndex

* Added support for snapshots and persistence

* Updated lockfile

* clippy

* Updated CHANGELOG

* Fixed issue with unwrapping snapshot.every

* Updated default config example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant