DO NOT MERGE!: feat(spider-storage): Cache-layer prototyping.

[LinZhihao-723]

Description

This PR should NOT be merged!

This PR contains the prototype of the cache layer implementation.

The core infra, including the task graph and job control, is implemented manually with a carefully designed locking protocol.

The factory function, which converts a serialized task graph into a job control, is implemented by AI.

The testing infra is implemented by AI, which provides a simulated scheduler system to test the correctness of the cache behavior, as long as collecting latency stats in terms of task instance registration and result writeback.

Run tests with the following command to show the stats:

cargo test -p spider-storage --lib --release -- --nocapture

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: d430a958-9487-44b6-9509-1c8a2946cb9a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

CodeRabbit can enforce grammar and style rules using `languagetool`.

Configure the reviews.tools.languagetool setting to enable/disable rules and categories. Refer to the LanguageTool Community to learn more.