Skip to content

HDDS-11476. Implement lesser/greater operation for --filter option of ldb scan command #7222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sumitagrawl merged 4 commits into from
Oct 8, 2024
Merged

HDDS-11476. Implement lesser/greater operation for --filter option of ldb scan command #7222

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
62508d4
Implement lesser/greater operation for --filter option of ldb scan co…
Sep 20, 2024
2f6a8f6
Add tests
Sep 20, 2024
a6e415b
add negative test cases
Sep 23, 2024
cbf9f70
Add failure tests
Sep 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 37 additions & 4 deletions hadoop-ozone/dist/src/main/smoketest/debug/ozone-debug-ldb.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,12 @@ Write keys
Run Keyword if '${SECURITY_ENABLED}' == 'true' Kinit test user testuser testuser.keytab
Execute ozone sh volume create ${VOLUME}
Execute ozone sh bucket create ${VOLUME}/${BUCKET} -l OBJECT_STORE
Execute dd if=/dev/urandom of=${TEMP_DIR}/${TESTFILE} bs=100000 count=15
Execute ozone sh key put ${VOLUME}/${BUCKET}/${TESTFILE}1 ${TEMP_DIR}/${TESTFILE}
Execute ozone sh key put ${VOLUME}/${BUCKET}/${TESTFILE}2 ${TEMP_DIR}/${TESTFILE}
Execute ozone sh key put ${VOLUME}/${BUCKET}/${TESTFILE}3 ${TEMP_DIR}/${TESTFILE}
Execute dd if=/dev/urandom of=${TEMP_DIR}/${TESTFILE}1 bs=100 count=10
Execute ozone sh key put ${VOLUME}/${BUCKET}/${TESTFILE}1 ${TEMP_DIR}/${TESTFILE}1
Execute dd if=/dev/urandom of=${TEMP_DIR}/${TESTFILE}2 bs=100 count=15
Execute ozone sh key put ${VOLUME}/${BUCKET}/${TESTFILE}2 ${TEMP_DIR}/${TESTFILE}2
Execute dd if=/dev/urandom of=${TEMP_DIR}/${TESTFILE}3 bs=100 count=20
Execute ozone sh key put ${VOLUME}/${BUCKET}/${TESTFILE}3 ${TEMP_DIR}/${TESTFILE}3
Execute ozone sh key addacl -a user:systest:a ${VOLUME}/${BUCKET}/${TESTFILE}3

*** Test Cases ***
Expand Down Expand Up @@ -71,6 +73,8 @@ Test ozone debug ldb scan
Should not contain ${output} objectID
Should not contain ${output} dataSize
Should not contain ${output} keyLocationVersions

Test ozone debug ldb scan with filter option success
# test filter option with one filter
${output} = Execute ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="keyName:equals:testfile2"
Should not contain ${output} testfile1
Expand All @@ -91,3 +95,32 @@ Test ozone debug ldb scan
Should not contain ${output} testfile1
Should not contain ${output} testfile2
Should not contain ${output} testfile3
# test filter option for size > 1200
${output} = Execute ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="dataSize:greater:1200"
Should not contain ${output} testfile1
Should contain ${output} testfile2
Should contain ${output} testfile3
# test filter option for size < 1200
${output} = Execute ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="dataSize:lesser:1200"
Should contain ${output} testfile1
Should not contain ${output} testfile2
Should not contain ${output} testfile3
# test filter option with no records match both filters
${output} = Execute ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="dataSize:lesser:1200,keyName:equals:testfile2"
Should not contain ${output} testfile1
Should not contain ${output} testfile2
Should not contain ${output} testfile3

Test ozone debug ldb scan with filter option failure
# test filter option with invalid operator
${output} = Execute ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="dataSize:lesserthan:1200"
Should contain ${output} Error: Invalid operator
# test filter option with invalid format
${output} = Execute And Ignore Error ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="dataSize:1200"
Should contain ${output} Error: Invalid format
# test filter option with invalid field
${output} = Execute And Ignore Error ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="size:equals:1200"
Should contain ${output} Error: Invalid field
# test filter option for lesser/greater operator on non-numeric field
${output} = Execute And Ignore Error ozone debug ldb --db=/data/metadata/om.db scan --cf=keyTable --filter="keyName:lesser:k1"
Should contain ${output} only on numeric values
24 changes: 24 additions & 0 deletions hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/debug/TestLDBCli.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,30 @@ private static Stream<Arguments> scanTestCases() {
Named.of("Filter invalid key", Arrays.asList("--filter", "keyName:equals:key9")),
Named.of("Expect key1-key3", null)
),
Arguments.of(
Named.of(KEY_TABLE, Pair.of(KEY_TABLE, false)),
Named.of("Default", Pair.of(0, "")),
Named.of("Filter dataSize<2000", Arrays.asList("--filter", "dataSize:lesser:2000")),
Named.of("Expect key1-key5", Pair.of("key1", "key6"))
),
Arguments.of(
Named.of(KEY_TABLE, Pair.of(KEY_TABLE, false)),
Named.of("Default", Pair.of(0, "")),
Named.of("Filter dataSize<500", Arrays.asList("--filter", "dataSize:lesser:500")),
Named.of("Expect empty result", null)
),
Arguments.of(
Named.of(KEY_TABLE, Pair.of(KEY_TABLE, false)),
Named.of("Default", Pair.of(0, "")),
Named.of("Filter dataSize>500", Arrays.asList("--filter", "dataSize:greater:500")),
Named.of("Expect key1-key5", Pair.of("key1", "key6"))
),
Arguments.of(
Named.of(KEY_TABLE, Pair.of(KEY_TABLE, false)),
Named.of("Default", Pair.of(0, "")),
Named.of("Filter dataSize>2000", Arrays.asList("--filter", "dataSize:greater:2000")),
Named.of("Expect empty result", null)
),
Arguments.of(
Named.of(BLOCK_DATA + " V3", Pair.of(BLOCK_DATA, true)),
Named.of("Default", Pair.of(0, "")),
Expand Down
Loading