[Filestore] implement blob compression#2777

Draft

SvartMetal wants to merge 24 commits intomainfrom users/svartmetal/issue-9/4

+908-150

Commits on Jan 1, 2025

add TBlobCompressionInfo to TMixedBlobMeta

Mikhail Montsev
authored and
SvartMetal
committed
compress blobs during compaction

Mikhail Montsev
authored and
SvartMetal
committed
add TBlobCompressionInfo to IMixedBlocksVisitor

Mikhail Montsev
authored and
SvartMetal
committed
simplify ReadBlob actor

Mikhail Montsev
authored and
SvartMetal
committed
use TBlobCompressionInfo::BlobCompressed() method instead of operator bool

Mikhail Montsev
authored and
SvartMetal
committed
code is simpler if TReadBlobActor uses TVector<TReadBlob::TBlock> instead of TVector<ui32> of block offsets

Mikhail Montsev
authored and
SvartMetal
committed
use c_str instead of data in TReadBlobActor

Mikhail Montsev
authored and
SvartMetal
committed
extract ReadUncompressedResponse function from TReadBlobActor::HandleGetResult

Mikhail Montsev
authored and
SvartMetal
committed
use TEvGetQueryInfo::BlockOffsets instead of TReadBlobRequest::Blocks

Mikhail Montsev
authored and
SvartMetal
committed
minor tweak

Mikhail Montsev
authored and
SvartMetal
committed
Use TReadBlob::TBlock instead of block offset (ui64) in TEvGetQueryInfo

Mikhail Montsev
authored and
SvartMetal
committed
pass TBlobCompressionInfo to TReadBlobRequest

Mikhail Montsev
authored and
SvartMetal
committed
further refactoring: rename TEvGetQueryInfo to TQuery and Offset, Size to use it for decompression

Mikhail Montsev
authored and
SvartMetal
committed
decompress data in TReadBlobActor

Mikhail Montsev
authored and
SvartMetal
committed
get rid of TQuery::Compressed flag

Mikhail Montsev
authored and
SvartMetal
committed
introduce TBlobCompressionInfo::TImpl

Mikhail Montsev
authored and
SvartMetal
committed
TBlobCompressionInfo should use shared_ptr instead of unique_ptr

Mikhail Montsev
authored and
SvartMetal
committed
TBlobCompressionInfo should store encoded data using TByteVector

Mikhail Montsev
authored and
SvartMetal
committed
split TUncompressedRange and TCompressedRange
SvartMetal
committed
add EAllocatorTag::BlobCompressionInfo
SvartMetal
committed
fix blob size after compression; implement 'identity' compression algorithm (needed to check correctness of the refactoring)
SvartMetal
committed
add service-kikimr-compression-test
SvartMetal
committed
fix bug and implement simple 'whole blob' compression algorithm
SvartMetal
committed
FlushBytes should respect TBlobCompressionInfo; fix subtle bug with missing TBlobCompressionInfo in UpdateBlockLists
SvartMetal
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Filestore] implement blob compression#2777

Commits on Jan 1, 2025

add TBlobCompressionInfo to TMixedBlobMeta

compress blobs during compaction

add TBlobCompressionInfo to IMixedBlocksVisitor

simplify ReadBlob actor

use TBlobCompressionInfo::BlobCompressed() method instead of operator bool

code is simpler if TReadBlobActor uses TVector<TReadBlob::TBlock> instead of TVector<ui32> of block offsets

use c_str instead of data in TReadBlobActor

extract ReadUncompressedResponse function from TReadBlobActor::HandleGetResult

use TEvGetQueryInfo::BlockOffsets instead of TReadBlobRequest::Blocks

minor tweak

Use TReadBlob::TBlock instead of block offset (ui64) in TEvGetQueryInfo

pass TBlobCompressionInfo to TReadBlobRequest

further refactoring: rename TEvGetQueryInfo to TQuery and Offset, Size to use it for decompression

decompress data in TReadBlobActor

get rid of TQuery::Compressed flag

introduce TBlobCompressionInfo::TImpl

TBlobCompressionInfo should use shared_ptr instead of unique_ptr

TBlobCompressionInfo should store encoded data using TByteVector

split TUncompressedRange and TCompressedRange

add EAllocatorTag::BlobCompressionInfo

fix blob size after compression; implement 'identity' compression algorithm (needed to check correctness of the refactoring)

add service-kikimr-compression-test

fix bug and implement simple 'whole blob' compression algorithm

FlushBytes should respect TBlobCompressionInfo; fix subtle bug with missing TBlobCompressionInfo in UpdateBlockLists