Add policy to decide when to delete old data

[mangalaman93]

[eeliu]

@mangalaman93

Thanks your bigqueue!

Do you have any plan to support deleting the old files? It looks stale for quite long time.

[mangalaman93]

I can work on it if your use case needs it. Could you tell me about your use case? How much do you have in the bigqueue? What kind of policy would be reasonable for your use case?

[eeliu]

@mangalaman93

Thanks your response, I added these features and customized some APIs under a very tight deadline.

Here is the changes: https://github.com/eeliu/bigqueue/blob/dev/grandecola_diff.md

PRs:

• refactor: update logger interface and methods for structured logging eeliu/bigqueue#16
• Add BacklogBytes API for O(1) queue backlog size measurement eeliu/bigqueue#10
• Pull Request: Arena Cleanup Mechanism, Map-based Refactor, and Reliability Fixes eeliu/bigqueue#12

If you find these APIs useful, I would be happy to submit the code to your repository to facilitate better long-term management.
I really appreciate you making this project open source. It helped me bring my ideas to life much faster.

[mangalaman93]

Could you create GC related changes in a PR and let's get them merged? I haven't seen all the changes but looks exciting.

[eeliu]

@mangalaman93

My pleasure, maybe this week, but not sure.

[eeliu]

#98 please review it, if you have time.