Skip to content

Add a synchronous "before checkpoint" hook? #12

@bjchambers

Description

@bjchambers

For some use cases of a write ahead log, I'd like to keep an in-memory representation (which gets recovered) of the contents of the log. When a checkpoint is made, I can drop the check-pointed information from memory since it is now part of the checkpoint.

One difficulty, as far as I can tell, is that the checkpoint method is called in a separate thread -- this means the following could happen:

  1. Commit some data A to the log
  2. Initiate a checkpoint
  3. Commit some data B to the log
  4. Actually perform the checkpoint

The problem I have is that at that point, the contents of A and B are in the in-memory representaiton, while I only want to checkpoint (and potentially drop) A. For my use case, I'm fine keeping the in-memory representation behind a Mutex (there should be only one writer to the log), which also simplifies a lot of the logic.

There are a few things that could help me:

  1. If the result of committing data indicated that a checkpoint would be taken. I think this would only help if there was a lock, but then I could detect that a checkpoint would be taken after A, and arrange to have a separate in-memory representation started for data added after that.
  2. If there was a "pre-checkpoint" hook that was called before other commits were allowed. Again, this would allow me to split the in-memory representation for B.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions