Skip to content

feat(db): add db cluster logic and tests#338

Merged
deven96 merged 12 commits into
mainfrom
feat/replication-m1-db-integration
Jun 17, 2026
Merged

feat(db): add db cluster logic and tests#338
deven96 merged 12 commits into
mainfrom
feat/replication-m1-db-integration

Conversation

@jimezesinachi

Copy link
Copy Markdown
Collaborator

No description provided.

Signed-off-by: Jim Ezesinachi <ezesinachijim@gmail.com>
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

Test Results

125 tests   124 ✅  4m 59s ⏱️
 32 suites    0 💤
  4 files      1 ❌

For more details on these failures, see this check.

Results for commit c203746.

♻️ This comment has been updated with latest results.

Signed-off-by: Jim Ezesinachi <ezesinachijim@gmail.com>
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown

Benchmark Results

group                                                        main                                   pr
-----                                                        ----                                   --
predicate_query_with_index/size_100                          1.00      3.0±0.01µs        ? ?/sec    1.04      3.1±0.01µs        ? ?/sec
predicate_query_with_index/size_1000                         1.17     33.4±0.06µs        ? ?/sec    1.00     28.5±0.10µs        ? ?/sec
predicate_query_with_index/size_10000                        1.03    412.2±2.80µs        ? ?/sec    1.00    401.3±0.94µs        ? ?/sec
predicate_query_with_index/size_100000                       1.00      8.6±0.94ms        ? ?/sec    1.10      9.5±2.33ms        ? ?/sec
predicate_query_without_index/size_100                       1.07      7.0±0.01µs        ? ?/sec    1.00      6.6±0.02µs        ? ?/sec
predicate_query_without_index/size_1000                      1.00    100.4±0.80µs        ? ?/sec    1.03    103.8±0.15µs        ? ?/sec
predicate_query_without_index/size_10000                     1.04    793.5±4.81µs        ? ?/sec    1.00    760.9±3.38µs        ? ?/sec
predicate_query_without_index/size_100000                    1.19     24.1±0.93ms        ? ?/sec    1.00     20.2±1.04ms        ? ?/sec
store_batch_insertion_without_predicates/size_100            1.01    199.9±2.89µs        ? ?/sec    1.00    197.4±2.58µs        ? ?/sec
store_batch_insertion_without_predicates/size_1000           1.00  1313.4±25.62µs        ? ?/sec    1.02  1336.9±17.93µs        ? ?/sec
store_batch_insertion_without_predicates/size_10000          1.07     16.3±0.37ms        ? ?/sec    1.00     15.2±0.21ms        ? ?/sec
store_batch_insertion_without_predicates/size_100000         1.04    152.1±1.62ms        ? ?/sec    1.00    146.6±1.70ms        ? ?/sec
store_retrieval_linear_cosine_similarity/size_100            1.00     85.3±0.59µs        ? ?/sec    1.01     86.0±0.88µs        ? ?/sec
store_retrieval_linear_cosine_similarity/size_1000           1.01    733.7±6.38µs        ? ?/sec    1.00    723.1±7.08µs        ? ?/sec
store_retrieval_linear_cosine_similarity/size_10000          1.00      8.0±0.07ms        ? ?/sec    1.03      8.2±0.10ms        ? ?/sec
store_retrieval_linear_cosine_similarity/size_100000         1.00     87.9±0.77ms        ? ?/sec    1.01     89.1±0.90ms        ? ?/sec
store_retrieval_linear_dot_product/size_100                  1.01     85.4±0.49µs        ? ?/sec    1.00     84.9±0.55µs        ? ?/sec
store_retrieval_linear_dot_product/size_1000                 1.02    730.0±6.66µs        ? ?/sec    1.00   714.8±10.30µs        ? ?/sec
store_retrieval_linear_dot_product/size_10000                1.01      7.9±0.12ms        ? ?/sec    1.00      7.9±0.08ms        ? ?/sec
store_retrieval_linear_dot_product/size_100000               1.00     81.1±0.62ms        ? ?/sec    1.04     84.1±0.61ms        ? ?/sec
store_retrieval_linear_euclidean_distance/size_100           1.00     85.2±0.46µs        ? ?/sec    1.00     85.3±0.69µs        ? ?/sec
store_retrieval_linear_euclidean_distance/size_1000          1.01    725.8±5.88µs        ? ?/sec    1.00    722.1±6.91µs        ? ?/sec
store_retrieval_linear_euclidean_distance/size_10000         1.01      8.0±0.08ms        ? ?/sec    1.00      8.0±0.12ms        ? ?/sec
store_retrieval_linear_euclidean_distance/size_100000        1.00     82.9±0.68ms        ? ?/sec    1.02     84.4±0.99ms        ? ?/sec
store_retrieval_no_condition/size_100                        1.01     86.1±0.78µs        ? ?/sec    1.00     85.5±0.51µs        ? ?/sec
store_retrieval_no_condition/size_1000                       1.01    736.8±5.26µs        ? ?/sec    1.00    730.8±6.01µs        ? ?/sec
store_retrieval_no_condition/size_10000                      1.02      7.7±0.08ms        ? ?/sec    1.00      7.6±0.05ms        ? ?/sec
store_retrieval_no_condition/size_100000                     1.01     83.2±0.98ms        ? ?/sec    1.00     82.5±1.05ms        ? ?/sec
store_retrieval_non_linear_hnsw/size_100                     1.00    162.7±1.85µs        ? ?/sec    1.01    164.5±1.83µs        ? ?/sec
store_retrieval_non_linear_hnsw/size_1000                    1.01    506.0±4.66µs        ? ?/sec    1.00    502.3±2.21µs        ? ?/sec
store_retrieval_non_linear_hnsw/size_10000                   1.00      2.9±0.32ms        ? ?/sec    1.03      3.0±0.14ms        ? ?/sec
store_retrieval_non_linear_hnsw/size_100000                  1.00     17.8±0.52ms        ? ?/sec    1.08     19.2±0.59ms        ? ?/sec
store_retrieval_non_linear_kdtree/size_100                   1.00    180.9±0.68µs        ? ?/sec    1.00    180.3±0.68µs        ? ?/sec
store_retrieval_non_linear_kdtree/size_1000                  1.00   1181.8±2.59µs        ? ?/sec    1.00   1178.8±2.43µs        ? ?/sec
store_retrieval_non_linear_kdtree/size_10000                 1.00     14.2±0.29ms        ? ?/sec    1.02     14.4±0.19ms        ? ?/sec
store_retrieval_non_linear_kdtree/size_100000                1.02    169.0±2.11ms        ? ?/sec    1.00    166.3±1.65ms        ? ?/sec
store_sequential_insertion_without_predicates/size_100       1.00    280.6±0.75µs        ? ?/sec    1.01    282.0±1.61µs        ? ?/sec
store_sequential_insertion_without_predicates/size_1000      1.00      2.8±0.01ms        ? ?/sec    1.00      2.8±0.00ms        ? ?/sec
store_sequential_insertion_without_predicates/size_10000     1.00     27.7±0.12ms        ? ?/sec    1.00     27.7±0.07ms        ? ?/sec
store_sequential_insertion_without_predicates/size_100000    1.00    276.4±0.38ms        ? ?/sec    1.01    278.6±0.60ms        ? ?/sec

Comment thread ahnlich/db/src/server/handler.rs Outdated
client_handler: Arc<ClientHandler>,
task_manager: Arc<TaskManager>,
config: ServerConfig,
cluster: Option<ClusterRuntime>,

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ClusterRuntime would ideally always contain store_handler.... and we typically only ever want to access store_handler directly independent of cluster if cluster doesn't exist? Seems like we should have a cleaner

enum StoreRuntime {
    Standalone(StoreHandler), // perhaps a more befitting rename to handler/ runtime to be consistent
    Cluster(ClusterRuntime), 
}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually agree with this, I went in the direction of not changing the shape of Server, but this is the better direction long term

}

pub(crate) async fn list_stores_response(
cluster: Option<&ClusterRuntime>,

@deven96 deven96 Jun 9, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not particularly sold on having optional cluster and a reference to store_handler when cluster always contains handler... see above comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A'firm, I will change the shape now

Comment thread ahnlich/utils/src/server.rs
Comment thread ahnlich/db/src/tests/cluster_tests.rs

@deven96 deven96 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments

Comment thread ahnlich/db/src/server/handler.rs

fn apply(&mut self, data: &DbCommand) -> Result<DbResponse, StorageError<u64>> {
match data {
DbCommand::CreateStore(payload) => {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's probably a nice way to turn decode_payload into a macro so we dont have to manually pass that string but can extract it from the ty via stringify!

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like

macro_rules! decode_payload {
    ($ty:ty, $payload:expr) => {{
        <$ty>::decode($payload).map_err(|err| StorageError::IO {
            source: StorageIOError::read_state_machine(&std::io::Error::other(format!(
                "failed to decode {} raft payload: {}",
                stringify!($ty),
                err,
            ))),
        })
    }};
}

let params: query::CreateStore =
    decode_payload!(query::CreateStore, payload)?;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, will implement this

use std::any::{Any, TypeId};

pub(crate) fn map_client_write_error(
command_name: &str,

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another good use of macros to enforce not needing command name alongside the actual command

Comment thread ahnlich/db/Cargo.toml Outdated
prost.workspace = true
openraft.workspace = true
ahnlich-replication = { path = "../replication", version = "*" }
bitcode = { version = "0.6", features = ["serde"] }

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should move to general Cargo.toml... my suspicion is we will reuse elsewhere

Repository owner deleted a comment from claude Bot Jun 16, 2026
Repository owner deleted a comment from claude Bot Jun 16, 2026
Repository owner deleted a comment from github-actions Bot Jun 16, 2026
Change from automatic PR event trigger to comment-based trigger.
This is the documented, working pattern for Claude Code Action.

Users comment '@claude' on a PR to trigger a review.
Claude will read CLAUDE.md for review guidelines.

Testing this simpler approach first to verify CLAUDE.md integration works.
Repository owner deleted a comment from github-actions Bot Jun 17, 2026
@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown

Claude finished @deven96's task in 0s —— View job


I'll analyze this and get back to you.

Repository owner deleted a comment from github-actions Bot Jun 17, 2026
@deven96 deven96 merged commit c203746 into main Jun 17, 2026
6 of 8 checks passed
@deven96 deven96 deleted the feat/replication-m1-db-integration branch June 17, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants