Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for data block hash index for column families with user-defined timestamps #13283

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

brennan913
Copy link

@brennan913 brennan913 commented Jan 9, 2025

Summary

This patch consists of two changes.

  • Strip user-defined timestamps (UDTs) from user keys during data block hash index creation and lookup. This ensures that the same key with distinct timestamps will hash to the same location, preventing Get()s from retrieving outdated values for a given timestamp. This operation is a noop for column families that don't use UDTs (i.e., for which ts_sz is 0).

  • Add a new API, Comparator::KeysAreBytewiseComparableOtherThanTimestamp to enable use of BlockBasedTableOptions::kDataBlockBinaryAndHash for column families that use a timestamp-aware comparator with otherwise bytewise-comparable keys. This API provides semantics similar to the existing Comparator::CanKeysWithDifferentByteContentsBeEqual, except it only requires that the user key itself (not including timestamp) is bytwise comparable. This new API returns false by default, and should be overridden to return true for comparators that support these semantics. To use the data block hash index, a column family should use a comparator that overrides one of these APIs, and select BlockBasedTableOptions::kDataBlockBinaryAndHash.

Resolves issue #12100

Proposed Semantics

(Note that only one of KeysAreBytewiseComparableOtherThanTimestamp and CanKeysWithDifferentByteContentsBeEqual needs to be overriden)

Bytewise Comparable Timestamp Enabled CanKeysWithDifferentByteContentsBeEqual KeysAreBytewiseComparableOtherThanTimestamp Hash Index Enabled?
Yes Yes True True Yes
Yes No False True Yes
No Yes True False No
No No True False No

Test Plan

New unit test in db_with_timestamp_basic_test.cc based on repro from originally flagged issue
make clean && make all check -j$(nproc)
make jclean rocksdbjava jtest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants