upgrade to release 5.8.7 #8
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Closes facebook#2589 Differential Revision: D5431502 Pulled By: siying fbshipit-source-id: 8ebf8c87883daa9daa54b2303d11ce01ab1f6f75
Summary: Closes facebook#2590 Differential Revision: D5432539 Pulled By: siying fbshipit-source-id: 49902453bba3c95c1fb8354441b2198649e52bf4
Summary: Closes facebook#2591 Differential Revision: D5432696 Pulled By: siying fbshipit-source-id: a613230ab916de0b279a65ef429ede65460a8db2
Summary: CodeMod: Prefer `ADD_FAILURE()` over `EXPECT_TRUE(false)`, et cetera. The tautologically-conditioned and tautologically-contradicted boolean expectations/assertions have better alternatives: unconditional passes and failures. Reviewed By: Orvid Differential Revision: D5432398 Tags: codemod, codemod-opensource fbshipit-source-id: d16b447e8696a6feaa94b41199f5052226ef6914
Summary:
In gcc-7 the following is an error identified by -Werror=class-memaccess
In file included from ./table/get_context.h:14:0,
from db/version_set.cc:43:
./table/block.h: In constructor ‘rocksdb::BlockReadAmpBitmap::BlockReadAmpBitmap(size_t, size_t, rocksdb::Statistics*)’:
./table/block.h:73:53: error: ‘void* memset(void*, int, size_t)’ clearing an object of type ‘struct std::atomic<unsigned int>’ with no trivial copy-assignment; use value-initialization instead [-Werror=class-memaccess]
memset(bitmap_, 0, bitmap_size * kBytesPersEntry);
^
In file included from ./db/version_set.h:23:0,
from db/version_set.cc:12:
/toolchain/include/c++/8.0.0/atomic:684:12: note: ‘struct std::atomic<unsigned int>’ declared here
struct atomic<unsigned int> : __atomic_base<unsigned int>
^~~~~~~~~~~~~~~~~~~~
As a solution the default initializer can be applied in list context.
Signed-off-by: Daniel Black <[email protected]>
Closes facebook#2561
Differential Revision: D5398714
Pulled By: siying
fbshipit-source-id: d883fb88ec7535eee60d551038fe91f14488be36
Summary: Should be 1024, obviously :) Closes facebook#2592 Differential Revision: D5435269 Pulled By: ajkr fbshipit-source-id: c59338a3900798a4733f0b205e534f21215cf049
Summary: Fix column_family_test with LITE build. I need this patch to fix 5.6 branch. Closes facebook#2597 Differential Revision: D5437171 Pulled By: yiwu-arbug fbshipit-source-id: 88b9dc5925a6b47af10c1b41bc5b07c4251a84b5
Summary: This patch enables using PinnableSlice for RowCache, changes include not releasing the cache handle immediately after lookup in TableCache::Get, instead pass a Cleanble function which does Cache::RleaseHandle. Closes facebook#2492 Differential Revision: D5316216 Pulled By: maysamyabandeh fbshipit-source-id: d2a684bd7e4ba73772f762e58a82b5f4fbd5d362
Summary: Unfortunately we can't use -DNDEBUG yet since we don't properly exclude the test libraries/executables from the non-debug builds on non-MSVC platforms. Previously this was failing on Linux for every build type except `CMAKE_BUILD_TYPE=Debug`. Reverts a48a62d Closes facebook#2595 Differential Revision: D5436182 Pulled By: ajkr fbshipit-source-id: 062f07cc9ce06a073b66054722b27bac1890dca3
Summary: This will fix the error: "error: pathspec '2.2.fb.branch' did not match any file(s) known to git." Tested by manually sshing to sandcastle and running the command. Closes facebook#2599 Differential Revision: D5441130 Pulled By: maysamyabandeh fbshipit-source-id: a22fd6a52221471bafbba8990394b499535e5812
Summary: The new local branch specified with -b cannot be called master. Use tmp prefix to avoid name collision. Closes facebook#2600 Differential Revision: D5442944 Pulled By: maysamyabandeh fbshipit-source-id: 4a623d9b21d6cc01bee812b2799790315bdf5f6e
Summary: For forward_compatible_checkout_objs the local branch is already created in previous step. This patch avoid recreating it. This should address "fatal: A branch named '3.10.fb' already exists." errors. Closes facebook#2606 Differential Revision: D5443786 Pulled By: maysamyabandeh fbshipit-source-id: 69d5a67b87677429cf36e3a467bd114d341f3b9c
Summary: We don't need this format compatibility hack anymore. We should remove it to make things simpler. Closes facebook#2607 Differential Revision: D5444107 Pulled By: siying fbshipit-source-id: 7ef587dd0cacfc15a4083a137adba8e6bfddac7e
Summary: Closes facebook#2608 Differential Revision: D5444797 Pulled By: siying fbshipit-source-id: 690581d03f37822e059a16085088e8e2d8a45016
Summary: as titled Closes facebook#2601 Differential Revision: D5441618 Pulled By: ajkr fbshipit-source-id: 334b098d90caadd3d07167d051cfb4ae65db98e1
Summary: Closes facebook#2610 Differential Revision: D5445705 Pulled By: siying fbshipit-source-id: d5f97ffdf5bc18b853c3a106755aca96a650e428
Summary: Moving static AdaptationContext to outside function to bypass tsan's false report with static initializers. It is because with optimization enabled std::atomic is simplified to as a simple read with no locks. The existing lock produced by static initializer is __cxa_guard_acquire which is apparently not understood by tsan as it is different from normal locks (__gthrw_pthread_mutex_lock). This is a known problem with tsan: https://stackoverflow.com/questions/27464190/gccs-tsan-reports-a-data-race-with-a-thread-safe-static-local https://stackoverflow.com/questions/42062557/c-multithreading-is-initialization-of-a-local-static-lambda-thread-safe A workaround that I tried was to move the static variable outside the function. It is not a good coding practice since it gives global visibility to variable but it is a hackish workaround until g++ tsan is improved. Closes facebook#2598 Differential Revision: D5445281 Pulled By: yiwu-arbug fbshipit-source-id: 6142bd934eb5852d8fd7ce027af593ba697ed41d
Summary: I decided not even to keep it as an INFO-level log as it is too normal for compactions to be skipped due to locked input files. Removing logging here makes us consistent with how we treat locked files that weren't pulled in due to overlap. We may want some error handling on line 422, which should never happen when called by `LevelCompactionBuilder::PickCompaction`, as `SetupInitialFiles` skips compactions where overlap causes the output level to pull in locked files. Closes facebook#2617 Differential Revision: D5458502 Pulled By: ajkr fbshipit-source-id: c2e5f867c0a77c1812ce4242ab3e085b3eee0bae
Summary: This diff addresses two problems. Both problems cause us to miss scheduling desirable compactions. One side effect is compaction picking can spam logs, as there's no delay after failed attempts to pick compactions. 1. If a compaction pulled in a locked input-level file due to user-key overlap, we would not consider picking another file from the same input level. 2. If a compaction pulled in a locked output-level file due to user-key overlap, we would not consider picking any other compaction on any level. The code changes are dependent, which is why I solved both problems in a single diff. - Moved input-level `ExpandInputsToCleanCut` into the loop inside `PickFileToCompact`. This gives two benefits: (1) if it fails, we will try the next-largest file on the same input level; (2) we get the fully-expanded input-level key-range with which we can check for pending compactions in output level. - Added another call to `ExpandInputsToCleanCut` inside `PickFileToCompact`'s to check for compaction conflicts in output level. - Deleted call to `IsRangeInCompaction` in `PickFileToCompact`, as `ExpandInputsToCleanCut` also correctly handles the case where original output-level files (i.e., ones not pulled in due to user-key overlap) are pending compaction. Closes facebook#2615 Differential Revision: D5454643 Pulled By: ajkr fbshipit-source-id: ea3fb5477d83e97148951af3fd4558d2039e9872
Summary: In this test we are deleting 100 files, and we are expecting DeleteScheduler to delete 26 files in the background and 74 files immediately in the foreground The main purpose of the test is to make sure that we delete files in foreground thread, which is verified in line 546 But sometimes we may end up with 26 files or 25 files in the trash directory because the background thread may be slow and not be able to delete the first file fast enough, so sometimes this test fail. Remove ``` ASSERT_EQ(CountFilesInDir(trash_dir_), 25); ``` Since it does not have any benefit any way Closes facebook#2618 Differential Revision: D5458674 Pulled By: IslamAbdelRahman fbshipit-source-id: 5556a9edfa049db71dce80b8e6ae0fdd25e1e74e
Summary: Remove some of the per-key logging by blob db to reduce noise. Closes facebook#2587 Differential Revision: D5429115 Pulled By: yiwu-arbug fbshipit-source-id: b89328282fb8b3c64923ce48738c16017ce7feaf
Summary: Seems the only function of the script is to create a new branch, which can be done easily. I'm removing it. Closes facebook#2623 Differential Revision: D5468681 Pulled By: yiwu-arbug fbshipit-source-id: 87dea5ecc4c85e06941ccbc36993f7f589063878
Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes facebook#2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
Summary: This uses `clang-tidy` to comment out unused parameters (in functions, methods and lambdas) in fbcode. Cases that the tool failed to handle are fixed manually. Reviewed By: igorsugak Differential Revision: D5454343 fbshipit-source-id: 5dee339b4334e25e963891b519a5aa81fbf627b2
Summary: This reverts the previous commit 1d7048c, which broke the build. Did a `git revert 1d7048c`. Closes facebook#2627 Differential Revision: D5476473 Pulled By: sagar0 fbshipit-source-id: 4756ff5c0dfc88c17eceb00e02c36176de728d06
Summary: The previous implementation of caching `file_size` index made no sense. It only remembered the original span of locked files starting from beginning of `file_size`. We should remember the index after all compactions that have been considered but rejected. This will reduce the work we do while holding the db mutex. Closes facebook#2624 Differential Revision: D5468152 Pulled By: ajkr fbshipit-source-id: ab92a4bffe76f9f174d861bb5812b974d1013400
Summary: Add and implement Iterator::Refresh(). When this function is called, if the super version doesn't change, update the sequence number of the iterator to the latest one and invalidate the iterator. If the super version changed, recreated the whole iterator. This can help users reuse the iterator more easily. Closes facebook#2621 Differential Revision: D5464500 Pulled By: siying fbshipit-source-id: f548bd35e85c1efca2ea69273802f6704eba6ba9
Summary: simply enable the macro in internal build, it wont hurt other sanitizers and will fix UBSAN issues Closes facebook#2625 Differential Revision: D5475897 Pulled By: IslamAbdelRahman fbshipit-source-id: 262c6fd5de3c1906f4b29e55b39110f125f41057
Summary: combining facebook#2568 and facebook#2612. Closes facebook#2620 Differential Revision: D5464394 Pulled By: IslamAbdelRahman fbshipit-source-id: 9f71d3058dd6adaf02ce3b2de3a81a1228009778
Summary: Previously setting `write_buffer_size` with `SetOptions` would only apply to new memtables. An internal user wanted it to take effect immediately, instead of at an arbitrary future point, to prevent OOM. This PR makes the memtable's size mutable, and makes `SetOptions()` mutate it. There is one case when we preserve the old behavior, which is when memtable prefix bloom filter is enabled and the user is increasing the memtable's capacity. That's because the prefix bloom filter's size is fixed and wouldn't work as well on a larger memtable. Closes facebook#3119 Differential Revision: D6228304 Pulled By: ajkr fbshipit-source-id: e44bd9d10a5f8c9d8c464bf7436070bb3eafdfc9
Summary: After move assignment, we need to re-initialized the moved PinnableSlice. Also update blob_db_impl.cc to not reuse the moved PinnableSlice since it is supposed to be in an undefined state after move. Closes facebook#3127 Differential Revision: D6238585 Pulled By: yiwu-arbug fbshipit-source-id: bd99f2e37406c4f7de160c7dee6a2e8126bc224e
Summary: To fix the issue of failing to decompress existing value after reopen DB with a different compression settings. Closes facebook#3142 Differential Revision: D6267260 Pulled By: yiwu-arbug fbshipit-source-id: c7cf7f3e33b0cd25520abf4771cdf9180cc02a5f
Summary: `compression` shadow the method name in `BlobFile`. Rename it. Closes facebook#3148 Differential Revision: D6274498 Pulled By: yiwu-arbug fbshipit-source-id: 7d293596530998b23b6b8a8940f983f9b6343a98
Summary: A race condition will happen when: * a user thread writes a value, but it hits the write stop condition because there are too many un-flushed memtables, while holding blob_db_impl.write_mutex_. * Flush is triggered and call flush begin listener and try to acquire blob_db_impl.write_mutex_. Fixing it. Closes facebook#3149 Differential Revision: D6279805 Pulled By: yiwu-arbug fbshipit-source-id: 0e3c58afb78795ebe3360a2c69e05651e3908c40
Summary: The current implementation of PinnableSlice move assignment have an issue facebook#3163. We are moving away from it instead of try to get the move assignment right, since it is too tricky. Closes facebook#3164 Differential Revision: D6319201 Pulled By: yiwu-arbug fbshipit-source-id: 8f3279021f3710da4a4caa14fd238ed2df902c48
Summary: Add -DPORTABLE=1 port::cacheline_aligned_alloc() has arguments swapped which prevents every single test from running. Closes facebook#2815 Differential Revision: D5751661 Pulled By: siying fbshipit-source-id: e0857d6e138ec46035b3c23d7c3c751901a0a4a0
Summary: Reuse WITH_JEMALLOC option in preparation for module search unification. Move jemalloc overrides into a separate .cc Remote obsolete JEMALLOC_NOINIT option. Closes facebook#3078 Differential Revision: D6174826 Pulled By: yiwu-arbug fbshipit-source-id: 9970a0289b4490272d15853920d9d7531af91140
Summary: This is a simpler version of facebook#3097 by removing all unrelated changes. Fixing the bug where concurrent writes may get Status::OK while it actually gets IOError on WAL write. This happens when multiple writes form a write batch group, and the leader get an IOError while writing to WAL. The leader failed to pass the error to followers in the group, and the followers end up returning Status::OK() while actually writing nothing. The bug only affect writes in a batch group. Future writes after the batch group will correctly return immediately with the IOError. Closes facebook#3201 Differential Revision: D6421644 Pulled By: yiwu-arbug fbshipit-source-id: 1c2a455c5b73f6842423785eb8a9dbfbb191dc0e
|
LGTM |
1 similar comment
|
LGTM |
Summary: it turns out that, with older GCC shipped from centos7, the SSE42 intrinsics are not available even with "target" specified. so we need to pass "-msse42" for checking compiler's sse4.2 support and for building crc32c.cc which uses sse4.2 intrinsics for crc32. Signed-off-by: Kefu Chai <[email protected]> Closes facebook#2950 Differential Revision: D6032298 Pulled By: siying fbshipit-source-id: 124c946321043661b3fb0a70b6cdf4c9c5126ab4
|
LGTM |
hx235
pushed a commit
to tabokie/rocksdb
that referenced
this pull request
Dec 27, 2023
Summary: I must have chosen trimming before frame 8 based on assertion failures, but that trims too many frame for a general segfault. So this changes to start printing at frame 4, as in this example where I've seeded a null deref: ``` Received signal 11 (Segmentation fault) Invoking LLDB for stack trace... Process 873208 stopped * thread tikv#1, name = 'db_stress', stop reason = signal SIGSTOP frame #0: 0x00007fb1fe8f1033 libc.so.6`__GI___wait4(pid=873478, stat_loc=0x00007fb1fb114030, options=0, usage=0x0000000000000000) at wait4.c:30:10 thread tikv#2, name = 'rocksdb:low', stop reason = signal SIGSTOP frame #0: 0x00007fb1fe8972a1 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12 Executable module set to "/data/users/peterd/rocksdb/db_stress". Architecture set to: x86_64-unknown-linux-gnu. True frame tikv#4: 0x00007fb1fe844540 libc.so.6`__restore_rt at libc_sigaction.c:13 frame tikv#5: 0x0000000000608514 db_stress`rocksdb::StressTest::InitDb(rocksdb::SharedState*) at db_stress_test_base.cc:345:18 frame tikv#6: 0x0000000000585d62 db_stress`rocksdb::RunStressTestImpl(rocksdb::SharedState*) at db_stress_driver.cc:84:17 frame tikv#7: 0x000000000058dd69 db_stress`rocksdb::RunStressTest(shared=0x00006120000001c0) at db_stress_driver.cc:266:34 frame tikv#8: 0x0000000000453b34 db_stress`rocksdb::db_stress_tool(int, char**) at db_stress_tool.cc:370:20 ... ``` Pull Request resolved: facebook#12101 Test Plan: manual (see above) Reviewed By: ajkr Differential Revision: D51593217 Pulled By: pdillinger fbshipit-source-id: 4a71eb8e516edbc32e682f9537bc77d073a7b4ed
mittalrishabh
pushed a commit
to mittalrishabh/rocksdb
that referenced
this pull request
Apr 1, 2024
…/) will be synced (tikv#8) Co-authored-by: Andrew Kryczka <[email protected]>
mittalrishabh
pushed a commit
to mittalrishabh/rocksdb
that referenced
this pull request
Apr 8, 2024
…/) will be synced (tikv#8) Co-authored-by: Andrew Kryczka <[email protected]>
mittalrishabh
added a commit
to mittalrishabh/rocksdb
that referenced
this pull request
Apr 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.