Help: Understanding Grace period?

[osseonews]

Sorry, if this sounds like a silly question, but I don't understand how the grace period works. If we are caching something in Redis and set a TTL, I assume this uses the TTL in Redis and so the Redis key expires in the TTL. So then how does the Grace period extend that? Or is that when you use the Grace period, the actual TTL value in Redis is set to the Grace value and not the TTL? If so, what is the point of the TTL exactly and how is it stored? I am tempted to think Grace is about in memory storage, but how can something be stored in memory so long in Node? usually, in my experience after a request, subsequent requests will get memory for about 5 minutes. So what does the Grace accomplish, the value will be evicted from memory already anyway, so where would the Grace actually be applicable?

[thisisnkc]

Good question. The important thing to understand is that BentoCache actually works with two different TTLs.

1. Logical TTL

This is the ttl you set. It determines when the cached value should be considered stale.

2. Physical TTL

This comes from the grace duration. It determines how long the key actually stays in storage before it is deleted.

Example

If you configure:

ttl: '10m'
grace: '6h'                                                                                                                                                                                 

BentoCache will store the key with a TTL of 6 hours (in both memory and Redis). At the same time, it embeds a logicalExpiration timestamp inside the stored value (current time + 10 minutes).

After 10 minutes, the value becomes logically expired, but it is still physically present in storage.

What happens after logical expiration?

When a request comes in after the logical TTL has passed:

1. BentoCache attempts to refresh the value by running your factory function.
2. The grace period does not skip this refresh attempt.
3. If the factory succeeds, the cache gets updated.
4. If the factory fails (for example, your database is down), BentoCache returns the stale value instead of throwing an error.

This is the main purpose of the grace period is allowing stale data to be served if refreshing the cache fails.

What about the in-memory cache (L1)?

Grace works at both the in-memory (L1) and Redis (L2) layers. Both store the value using the grace duration as the physical TTL and check logicalExpiration on reads.

In a long-running Node.js server, the in-memory cache is not short-lived, it persists for the lifetime of the process. So a grace: '6h' entry genuinely lives in memory for 6 hours and can serve as a stale fallback there too, without even hitting Redis.

The "memory is short-lived" scenario only applies to serverless/edge functions. For a regular server, L1 and L2 grace are equally meaningful.

[osseonews]

Thank you very much for this explanation! I guess my question now is what is the benefit of the Logical TTL, which I assume lives in memory. Why not just call the Physical TTL in order (memory first, and then Redis, if memory doesn't exist) and if the value exists there use it and if not run the factory function? Presumably the most "expensive" operation here is the factory function, so what purpose does it serve to hit this so many times with a Low Logical TTL. If the argument is that you don't want to hit memory or Redis often, well you are doing that anyway with Logical TTL and Redis is designed for this type of thing. To me the Logical TTL just seems unnecessary complexity here. I would think the idea should just be, run a factory function, save the data in Memory and Redis and then call those in order to get the data. If both fail, you run the factory function again.

Also, it seems like you can already set a multi tier cache: https://bentocache.dev/docs/multi-tier - so what happens when you do a Grace period here with multi-tier? Are 2 keys with different expirations set in each storage cache? So for example, you have a key in Redis with the TTL and then another key with the Grace Period? If not, then what does the TTL do in the multi-tier?

PS Thanks for explaining about in-memory. I mainly have worked in server less, so I guess that's why I never realized that in-memory can last so long.