[pull] memtable_as_log_index from topling:memtable_as_log_index#4
Open
pull[bot] wants to merge 102 commits into
Open
[pull] memtable_as_log_index from topling:memtable_as_log_index#4pull[bot] wants to merge 102 commits into
pull[bot] wants to merge 102 commits into
Conversation
This makes ToplingDB Web auto refresh pretty
rockeet
force-pushed
the
memtable_as_log_index
branch
2 times, most recently
from
9b42dd6 to
5c00a65
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
5c00a65 to
68fbc5f
Compare
And strip db_bench in build-trial.sh
rockeet
force-pushed
the
memtable_as_log_index
branch
from
538e4a3 to
b0622bb
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
b0622bb to
1aa07fa
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
70b0c77 to
8a40448
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
8a40448 to
355304b
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
from
e4ecd25 to
0892907
Compare
Allow closing a single DB instance managed by the repo.
rockeet
force-pushed
the
memtable_as_log_index
branch
from
f07d1df to
207a070
Compare
rockeet
force-pushed
the
memtable_as_log_index
branch
2 times, most recently
from
720d183 to
80f7c95
Compare
These two methods let users keep their existing DBOptions and CFOptions construction code while selectively applying options from a SidePluginRepo YAML/JSON config. Users load a config file into a SidePluginRepo, then call side_plugin_db_options_update_from(opt, repo, "name"); side_plugin_cf_options_update_from(opt, repo, "name"); to overlay the repo's named options onto hand-crafted Options objects. This migration path preserves all ToplingDB engine optimizations (MemTable, SST formats, etc.) but foregoes SidePlugin features that depend on full SidePluginRepo-managed lifecycle: Web UI observability, Prometheus integration, online config hot-update, and distributed compaction.
rockeet
force-pushed
the
memtable_as_log_index
branch
from
80f7c95 to
bf54b9c
Compare
Also override CFOptions in CreateColumnFamily, override options through MaybeOptionsUpdateFrom() If import the conf file failed, the process dies immediately
Under TOPLINGDB_EASY_MIGRATE_CONF, db-path components are treated as
a namespace hierarchy for config lookup, from most specific (full path)
to most general ("default"). CFOptions now follow the same path-prefix
walk with bare-name and "default" fallbacks that DBOptions always had.
Design rationale: https://github.com/topling/rockside/wiki/Easy-Migrate-Without-Code-Change
rockeet
force-pushed
the
memtable_as_log_index
branch
from
b1128dc to
18084ab
Compare
- Call MaybeRetainDB via ROCKSDB_SCOPE_EXIT after successful DBImpl::Open, ReadOnly, and Secondary opens. - Call MaybeForgetDB at the start of ~DBImpl. - Clear *dbptr before attaching scope exit to avoid retaining on early validation failure; setting *dbptr = nullptr here also fixes an obscure upstream bug where the output pointer could be left uncleared when validation fails before the later assignment. - Bump rockside for EasyMigrate HTTP gating and retain/forget implementation.
The MaybeRetainDB fix in the rockside submodule needs to read persist_stats_cf_handle_ from a DB* pointer, but it is a private member with no accessor. Add a public accessor and a wrapper function, and update the submodule to use them. The submodule fix replaces dangling default_cfh and stat_cfh handles with live pointers before storing them in the sideplugin repo, since RocksDB deletes the original handles on simple Open.
Track column family lifecycle in the sideplugin repo during easy migrate: register CF in CreateColumnFamily via scope-exit guard, and unregister in DropColumnFamily before the handle becomes invalid.
Pulls in the yaml config change and the MaybeRetainCF/MaybeForgetCF implementation in the submodule.
Expose ReadOptions::internal_is_in_pinning_section through the C API, needed by Rust bindings for ReadOptionsScopePinIfNotPinned guard.
rockeet
force-pushed
the
memtable_as_log_index
branch
from
f5bd243 to
5d68e7e
Compare
…ndex option When DB Open replays WAL files, the reader was unconditionally configured based on the current memtable_as_log_index option. If the option changed between runs (e.g. config file or env var), the WAL records would be parsed with the wrong header format, causing CRC mismatch and recovery failure. Now IsMemTableAsLogIndexFile() probes the actual on-disk format of each WAL file before replay, and the reader uses the detected format regardless of the current option setting. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously WAL format was detected unconditionally via CRC32 self- consistency on every DB open, which carries a 1/2^32 false-positive risk per file. This risk is only relevant when memtable_as_log_index was actually changed between runs — an extremely rare event. Gate the detection behind DBOptions::check_wal_format (default false) so users only pay the risk when they know a format switch occurred. This reduces false-positive exposure by orders of magnitude without requiring any WAL format changes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
rockeet
force-pushed
the
memtable_as_log_index
branch
from
609a616 to
8fa8e5d
Compare
…format is set When the user explicitly opts into check_wal_format, a detection I/O error should not be silently skipped — it must be surfaced so the user can correct the situation (e.g. revert memtable_as_log_index and retry). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…lure Ensures the log always records which file and what error caused the detection to fail, consistent across all call sites. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )