CNDB-16146: Reproduce missing rows failure#2143

Open

michaelsembwever wants to merge 1665 commits intomain-5.0from

mck-cndb-16146-main-5.0

This pull request is big! We're only showing the most recent 250 commits

Commits on May 17, 2025

CNDB-13305: change IndexComponentDiscovery#discoverComponents signature to use SSTableReader and add more loggings to index tracking (#1644)

jasonstack
authored and
djatnieks
committed
CNDB-12620 Users should get INDEX_BUILD_IN_PROGRESS instead of INDEX NOT AVAILABLE when they cannot query because an index is building.

authored and
djatnieks
committed
CNDB-13171: Fix query_filters guardrail using the index analyzer instead of the query analyzer (#1615)

adelapena
authored and
djatnieks
committed
HCD-62 Upgrade of several libraries (#1563)

bereng
authored and
djatnieks
committed
CNDB-13480: NullPointerException when using ORDER BY BM25 after deleting a row (#1653)

Enrico Olivelli
authored and
djatnieks
committed
CNDB-13353: Queries with index-based ordering should reject aggregation queries (#1638)

adelapena
authored and
djatnieks
committed
CNDB-13462: Add support for updating terms in SAI's TrieMemoryIndex (#1200)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13446: Make search-then-sort BM25 queries apply BM25 filter (#1659)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13464: Ordering on non-clustering column requires the column to be indexed with a non-analyzed index (#1656)

ekaterinadimitrova2
authored and
djatnieks
committed
CNDB-13457: Fix problems with byte-comparable versions in SAI (#1655)

blambov
authored and
djatnieks
committed
CNDB-13499: Optimize simple BM25 by deferring PrK creation (#1662)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13505: Update PR checklist reminding people about the correct license header in new files

ekaterinadimitrova2
authored and
djatnieks
committed
CNDB-13408 add BM25 tests to compare with CNDB (#1663)

k-rus
authored and
djatnieks
committed
CNDB-13417: Reduce object alloc. on brute force hybrid ann path (#1643)

michaeljmarshall
authored and
djatnieks
committed
Handle IndexBuildInProgress exception in InboundSink

ekaterinadimitrova2
authored and
djatnieks
committed
CNDB-13372 Introduce min Paxos backoff time (#1651)

aymkhalil
authored and
djatnieks
committed
CNDB-13618 remove long index name test after fix (#1681)

k-rus
authored and
djatnieks
committed
CNDB-12937: Update jvector to 4.0.0-beta.2; add new graph construction parameters to index config (#1676)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13624: Silence snakeyaml warnings in test-cdc by removing duplicate cdc_enabled key in base yaml (#1680)

jkni
authored and
djatnieks
committed
CNDB-13583: Add vector ann and brute force metrics (#1683)

michaeljmarshall
authored and
djatnieks
committed
CNDB-12937: enable jvector 4; remove jvector 3 write support (#1685)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13582: Ignore TOC when discovering SAI components of sstables from DSE

Piotr Kołaczkowski
authored and
djatnieks
committed
CC5 Fix CreateTableStatementCompactionStrategiesTest
djatnieks
committed
CC5 Fix CQLSSTableWriterClientTest.testWriteWithSAI
djatnieks
committed
CC5 Fix IndexNameTest.testQuotedAndLongColumnNames after port of CNDB-12503
djatnieks
committed
CNDB 13650: fix SSTableCorruptionDetectionTest.testSSTableScanner (#1690)

driftx
authored and
djatnieks
committed
CC5: IndexStatusManagerTest accomodates INDEX_BUILD_IN_PROGRESS

driftx
authored and
djatnieks
committed
CNDB-13401 Fix CQLSSTableWriterDaemon tests (#1682)
djatnieks
committed
CNDB-13118 Fix CoordinatorReadLatencyMetricTest changing internalPagingWIthAggregateTest to use paging in bytes for CC. (#1702)
djatnieks
committed
CNDB-13671 simplify tests with collection condition (#1688)

k-rus
authored and
djatnieks
committed
CNDB-13696 fix empty iterator access in BM25 search on partial SSTable (#1691)

k-rus
authored and
djatnieks
committed
CNDB-13785: fix IndexComponentDiscover#StateBuilder to pass MB instead of bytes (#1699)

jasonstack
authored and
djatnieks
committed
CNDB-13524 Adds a timeout to Future.get calls in PendingAntiCompactionTest (#1664)
djatnieks
committed
CC5 cleanup merge conflict markers after porting CNDB-13457
djatnieks
committed
CC5: move first delete to deleteWithConfirm, add debugging

driftx
authored and
djatnieks
committed
CC5: try a longer sleep to generate more contention in TrieMemtableMetricsTest

driftx
authored and
djatnieks
committed
CC5: don't confirm second delete in CommitLogSegmentManagerCDCTest

driftx
authored and
djatnieks
committed
CNDB-13696: Add BM25 hybrid search test on collections (#1701)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13766: Upgrade to jvector 4.0.0-beta.4 (#1698)

michaeljmarshall
authored and
djatnieks
committed
SAI: test that default stop words are the ENGLISH_STOP_WORDS_SET (#1697)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13718: Switch to official Apache Lucene 9.8.0 dep (#1694)

michaeljmarshall
authored and
djatnieks
committed

Commits on May 18, 2025

CNDB-13639: Adds optional prefetching for known sequential reads (#1692)

authored and
djatnieks
committed

Commits on May 21, 2025

CC5 fix CompactionStrategyStatisticsTest; since CASSANDRA-19596 both intervals in Interval.AsymetricOrdering are Comparable and the SSTableReader mock in BaseCompactionStrategyTest violated the Com…
djatnieks
committed

Commits on May 27, 2025

CC5 fix MixedIndexImplementationTest.shouldNotBeSelectedForCustomExpressions, changing default value of StorageAttachedIndexOptions.prioritize_over_legacy_index to 'true'.
djatnieks
committed

Commits on May 28, 2025

CNDB-13163: Fix incorrect data file length for early open sstables (#1707)

blambov
authored and
djatnieks
committed

Commits on May 29, 2025

CNDB-13689: use NodeQueue::pushMany to decrease time complexity to build heap (#1693)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13822: Add ANN_OPTION use_pruning (#1704)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13833: Fix QueryTimeoutTest on JDK22

ekaterinadimitrova2
authored and
djatnieks
committed
CNDB-13625: Implement rerankless vector search through ANN_OPTIONS (#1705)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13848: avoid serializing AlwaysPresentFilter when BF memory limit is reached (#1710)

jasonstack
authored and
djatnieks
committed
CNDB-13847: Add Token::fromLongValue method to simplify SAI token creation (#1709)

michaeljmarshall
authored and
djatnieks
committed
CNDB-13739 count numRows correctly in SSTable SAI (#1695)

k-rus
authored and
djatnieks
committed

Commits on Jun 3, 2025

CNDB-13830: Fix KD-tree metrics test (main-5.0) (#1779)
Piotr Kołaczkowski
authored

Commits on Jun 10, 2025

Commits on Jun 13, 2025

CNDB-13902: Fix incorrect results of min / max in-built functions on clustering columns in descending order - port CASSANDRA-20295

ekaterinadimitrova2
authored and
driftx
committed
Cherry-pick all the fixed about CNDB-13591 into main branch (#1719)

authored and
driftx
committed
Document recent vector index and query config options (#1708)

michaeljmarshall
authored and
driftx
committed
CNDB-13075: Reuse the analyzed tokens of the right operand of an analyzed expression (#1620)

adelapena
authored and
driftx
committed
CNDB-13693: Make SAI's view referenceable to simplify locking query view (#1700)

michaeljmarshall
authored and
driftx
committed
CNDB-13905: Fix valueCount check and error message in BKDWriter

Piotr Kołaczkowski
authored and
driftx
committed
CNDB-13905 provide total point count to BKDWriter

k-rus
authored and
driftx
committed
Improve code in affected files

k-rus
authored and
driftx
committed
CNDB-12407: lazily load token in PrimaryKeyWithSource (#1500)

michaeljmarshall
authored and
driftx
committed
CNDB-14488: Remove unused imports
driftx
committed
HCD-110 Add plugin support for CQLSH (#1722)

authored and
driftx
committed

Commits on Jun 16, 2025

HCD-151: Port CASSANDRA-20636 (5.0) (#1806)
szymon-miezal
authored

Commits on Jun 17, 2025

Commits on Jun 18, 2025

CNDB-14153: Fix SAI updates (non-null solution) (#1751)

michaeljmarshall
authored and
driftx
committed
CNDB-14171: handle update as insert for SAI version AA (#1748)

michaeljmarshall
authored and
driftx
committed
CNDB-14160: Fix IndexContext#getReferencedView (#1744)

michaeljmarshall
authored and
driftx
committed

Commits on Jul 1, 2025

CNDB-13666 use all documents in calculating average docs length (#1686)

k-rus
authored and
driftx
committed
CNDB-13978: Don't count tombstones in DataLimits.Counter.CQLCounter#bytesCounted (#1735)

authored and
driftx
committed
CNDB-14175: Create necessary tmp directories in SaiRandomizedTest before we run tests. The patch can be tested on InvertedIndexSearcherTest, for example.

ekaterinadimitrova2
authored and
driftx
committed
CNDB-14614-re-merge: abort compaction task or index build if index is unloaded after tenant unassignment (#1766)

jasonstack
authored and
driftx
committed
CNDB-141284: use `ByteBuffer.remaining` instead of `limit` to asset size (#1755)

pcmanus
authored and
driftx
committed
CNDB-14167: Set jvector_version on disk file format to 2 (#1757) (#1760)

authored and
driftx
committed
CNDB-14210: Fix analyzed sai index on compound partition key column (#1763)

michaeljmarshall
authored and
driftx
committed
CNDB-14232 fix flakiness in BM25 by createIndex (#1769)

k-rus
authored and
driftx
committed
cndb-14201: allow UCS#getNextBackgroundTasks to config custom compaction observer for composite compaction (#1767)

jasonstack
authored and
driftx
committed
CNDB-14182: Add some null checks in InMemoryTrie

Piotr Kołaczkowski
authored and
driftx
committed
CNDB-13770: Separate timeout for aggregation queries (#1740)

jakubzytka
authored and
driftx
committed
CNDB-14242: Upgrade jvector to 4.0.0-beta.5 (#1773)

michaeljmarshall
authored and
driftx
committed
CNDB-14057: Make SAI metrics accessible to CNDB

Piotr Kołaczkowski
authored and
driftx
committed

Commits on Jul 2, 2025

Commits on Jul 7, 2025

CNDB-13565: allow to set memtable shard lock fairness via JMX and a system property (#1785)

jakubzytka
authored and
driftx
committed

Commits on Jul 16, 2025

CNDB-13997 maintain total terms count in memindex (#1776)

k-rus
authored and
driftx
committed
CNDB-14291: Fix flaky SingleNodeQueryFailureTest (#1784)

Piotr Kołaczkowski
authored and
driftx
committed
CNDB-14159: Fix DOC_LENGTHS offset calculations (#1745)

michaeljmarshall
authored and
driftx
committed
HCD-153 C* 3.x don't like the trailing sha in the version over gossip (VersionedValue)

michaelsembwever
authored and
driftx
committed
CNDB-14374: Fix flaky test testIndexExceptionsTwoIndexesOn3NodeCluster (#1796)

ekaterinadimitrova2
authored and
driftx
committed
CNDB-14306: Update ANN_USE_SYNTHETIC_SCORE to default to true for SAI version >EC (#1787)

michaeljmarshall
authored and
driftx
committed
CNDB-14317: Optimize doc freq computation for memtable BM25 queries (#1789)

michaeljmarshall
authored and
driftx
committed

Commits on Jul 17, 2025

Commits on Jul 18, 2025

CNDB-14481: Fix IllegalStateException in SegmentMetadataBuilder (#1808)

Piotr Kołaczkowski
authored and
driftx
committed
CNDB-14392: Use QueryView's MemtableIndex references for search (#1799)

authored and
driftx
committed
CNDB-14477: Validate SAI Components in Verifier (#1816)

michaeljmarshall
authored and
driftx
committed
CNDB-14524: Reset view before calling GraphSearcher#search (#1826)

michaeljmarshall
authored and
driftx
committed
CNDB-14847 CC5 Restore SSTableFormat.requireComponents method (#1890)
djatnieks
authored
CNDB-14848 CC5 remove unused duplicated constants from SchemaConstants (#1891)
djatnieks
authored

Commits on Jul 21, 2025

Commits on Jul 23, 2025

CNDB-14501: Enable checksum for jvector index files starting at version EC (#1841)

michaeljmarshall
authored and
driftx
committed
HCD-130 incremental repair failure during compaction (#1728)

bereng
authored and
driftx
committed
CNDB-13553 aggregate document frequencies on entire node (#1802)

k-rus
authored and
driftx
committed
Remove unused test imports
driftx
committed
CNDB-10308: Fix flaky BinLogTest (#1840)

Piotr Kołaczkowski
authored and
driftx
committed
CNDB-14722: handle OOM error for compaction task (#1856)

jasonstack
authored and
driftx
committed
CNDB-14725: add SSTableFlushObserver#onSSTableWriterSwitched to flush SAI segment builder for written shards without waiting for entire transaction to complete (#1859)

jasonstack
authored and
driftx
committed
CNDB-13192: Bump the default `cassandra.sai.latest.version` from `DC` to `EC` so we support BM25 by default. (#1737)

ekaterinadimitrova2
authored and
driftx
committed
CNDB-14353: fetch non-compacting sstables instead of live sstables for UCS (#1861)

jasonstack
authored and
driftx
committed

Commits on Jul 24, 2025

CNDB-14797: Upgrade to jvector 4.0.0-rc1 (#1866)

michaeljmarshall
authored and
driftx
committed

Commits on Jul 28, 2025

Commits on Jul 30, 2025

DSP-23657: Introduce sstable encryption (v2) (#1795)

authored and
driftx
committed
HCD-111 Usability Requirements and Checks

michaelsembwever
authored and
driftx
committed
CNDB-12651: Fix flaky VectorDistributedTest.rangeRestrictedTest (#1834)

adelapena
authored and
driftx
committed
CNDB-13129: Add CqlBuilder.OptionsBuilder

Piotr Kołaczkowski
authored and
driftx
committed
CC5 fix CNDB-9443 to allow scheduled executors to be created with a specified number of threads instead of only single threaded. (#1933)
djatnieks
authored
HCD-147: Add createsystemkey subcommand to nodetool (#1759)

szymon-miezal
authored and
driftx
committed

Commits on Jul 31, 2025

Commits on Aug 1, 2025

Add test crc32 files to rat excludes
driftx
committed
CNDB-13129: Add index hints

adelapena
authored and
driftx
committed

Commits on Aug 12, 2025

CNDB-14577: Compact all SSTables of a level shard if their number reaches a limit (#1873)

authored and
djatnieks
committed

Commits on Aug 13, 2025

CNDB-14921: Add JMH benchmark for SAI queries selectivity (#1913)

adelapena
authored and
djatnieks
committed
HCD-178: Upgrade commons-lang3 from 3.11 to 3.18.0 (#1919)

szymon-miezal
authored and
djatnieks
committed
CNDB-14691 add DoD link to CNDB repo (#1857)

jtgrabowski
authored and
djatnieks
committed

Commits on Aug 14, 2025

CNDB-15098: port missing MetadataSerializer changes
driftx
committed

Commits on Aug 15, 2025

CNDB-14983: Fix MetadataSerializerTest#testSerializationWithEncryption under bigtable test profile. (#1930)

jkni
authored and
djatnieks
committed

Commits on Aug 18, 2025

CNDB-9018: cache SSTable:CompactionMetadata in order to not read from storage multiple times (#1922)

authored and
djatnieks
committed
CNDB-15002: Fix handling of grouped primary keys in ScoreOrderedResultRetriever (#1932)

michaeljmarshall
authored and
djatnieks
committed
CNDB-15001: Fix testing of ann_use_synthetic_score (#1931)

adelapena
authored and
djatnieks
committed

Commits on Aug 19, 2025

CND-14215: fix SSTablePartitionsTest.testDirectory
driftx
committed

Commits on Aug 21, 2025

CNDB-15097: Mock with stubOnly and clear mocks between runs to save heap
driftx
committed

Commits on Aug 22, 2025

CNDB-15177: fix GuardrailNonPartitionRestrictedQueryTest by restoring CQL truncation behavior
driftx
committed

Commits on Aug 26, 2025

CNDB-15038: set quoteValue to true to fix CqlBuilderTest
driftx
committed

Commits on Aug 28, 2025

Commits on Aug 29, 2025

Commits on Sep 3, 2025

CNDB-15262: add needsReconciliation flag to fix IntersectFilteringQueryTest
driftx
committed

Commits on Sep 4, 2025

CNDB-14207: Don't mark the index non-queryable if flush fails (#1770) (#1970)

djatnieks
and
Piotr Kołaczkowski
authored
CNDB-15022 Introduce additional guardrail profile for HCD (#1976)

djatnieks
and
michaelsembwever
authored
CNDB-13408 Improve BM25 test with better logic (#1963)

k-rus
authored and
djatnieks
committed

Commits on Sep 5, 2025

Commits on Sep 7, 2025

DSP-24600 & CASSANDRA-20485 Fixes & test additions for handling of UDTs across versions

authored and
djatnieks
committed

Commits on Sep 8, 2025

CNDB-14657: disable synthetic score for older versions in FeaturesVersionSupportTester
driftx
committed

Commits on Sep 11, 2025

Commits on Sep 15, 2025

CNDB-15332: set sasi_indexes_per_table_fail_threshold dbaas default to NO_LIMIT
driftx
committed

Commits on Sep 22, 2025

CNDB-15446 Fix GuardrailsConfigDefaultsTest with correct checks for page_size_fail_threshold and page_weight_fail_threshold
djatnieks
committed

Commits on Sep 25, 2025

CNDB-15432: CNDB-15155: Segregate SAI's query metrics per query type (#1969)

adelapena
authored and
michaelsembwever
committed

Commits on Sep 26, 2025

Commits on Sep 30, 2025

CNDB-15526: Load TOC component before performing component discovery (#2028)
driftx
authored

Commits on Oct 10, 2025

Commits on Oct 12, 2025

CNDB-15565: CNDB-15160: add counter to track num of skipped compaction aggregates due to insufficient disk space (#1962)

jasonstack
authored and
michaelsembwever
committed

Commits on Oct 14, 2025

CNDB-15482: CNDB-15435: Only count live data size in bytes-based paging (#2010)

adelapena
authored and
michaelsembwever
committed
CNDB-15483: CNDB-15300: Add `SSTableReader#getApproximatePositionsForRanges` (#1993)

pcmanus
authored and
michaelsembwever
committed
CNDB-15484: CNDB-14509: Mark function as public (#1995)

niksajakovljevic
authored and
michaelsembwever
committed

Commits on Oct 15, 2025

CNDB-15498: CNDB-15141: add LogTransaction#validate API to perform validation before txn prepare (#2008)

jasonstack
authored and
michaelsembwever
committed
CNDB-15500: CNDB-15452: Split SAI metrics query types into disjoint categories (#2015)

Piotr Kołaczkowski
authored and
michaelsembwever
committed
CNDB-15577: CNDB-14650: allow mixed mode repair

driftx
authored and
michaelsembwever
committed
CNDB-15694: CNDB-15485: Fix ResultRetriever key comparison to prevent dupes in result set (#2024)

michaeljmarshall
authored and
michaelsembwever
committed

Commits on Oct 16, 2025

Commits on Oct 17, 2025

CNDB-15716: register sensors in paxos v2 (#2070)
driftx
authored

Commits on Oct 20, 2025

CNDB-15447: set CQL timeout equal to server timeout in AggregationQueriesTest (#2073)
driftx
authored

Commits on Oct 21, 2025

CNDB-15672: SystemDistributedKeyspace uses executeInternal to bypass authorization (#2062)
driftx
authored

Commits on Oct 28, 2025

CNDB-15581: CDNB-15253: ninja-fix for `ant check`
michaelsembwever
committed

Commits on Nov 3, 2025

CNDB-15885 Remove apache/cassandra-5.0 from jvm-dtest upgrades
michaelsembwever
committed

Commits on Nov 4, 2025

CNDB-15843 To maintain thread-local consistency, capture and restore ExecutorLocals within Message.execute and Dispatcher.processInit. This ensures that Future callbacks, which may execute on diffe…
djatnieks
committed

Commits on Nov 10, 2025

CNDB-15911: Add sensor tracking for Paxos V2 operations (#2107)
driftx
authored

Commits on Nov 12, 2025

CNDB-15969 Remove duplicate SystemDistributedKeyspace calls missed during rebase if STAR-993 onto Apache Cassandra 5.0.4
djatnieks
committed
CNDB-15631: CNDB-8641: Add a metric to count all request errors (#1983)

cbornet
authored and
michaelsembwever
committed
CNDB-15632: CNDB-15157: limit log lines during commitlog replay (#2001)

driftx
authored and
michaelsembwever
committed
CNDB-15665: CNDB-14861: Fix usage of PrimaryKeyWithSource in SAI

michaeljmarshall
authored and
michaelsembwever
committed
CNDB-15666: CNDB-15570: Fix handling mixed key types in SAI iterators

Piotr Kołaczkowski
authored and
michaelsembwever
committed
CNDB-15667: CNDB-15623: Only use write path for CDC tables in CassandraStreamReceiver if CDC is enabled on the node (#2043)

jkni
authored and
michaelsembwever
committed

Commits on Nov 13, 2025

Commits on Nov 14, 2025

Commits on Nov 17, 2025

Commits on Nov 18, 2025

Commits on Nov 19, 2025

CNDB-15995 Add CC 4.0 schema backward compatibility mode (#2126)
djatnieks
authored

Commits on Dec 1, 2025

THROWAWAY: CNDB-16146: Reproduce missing rows failure
michaelsembwever
committed

Commits on Feb 2, 2026

Fix test directory handling
driftx
committed