fstree: serve `Stream` from the storage #3431

End-rey · 2025-06-27T21:30:57Z

codecov · 2025-06-27T21:33:33Z

Codecov Report

Attention: Patch coverage is 56.94444% with 31 lines in your changes missing coverage. Please review.

Project coverage is 21.17%. Comparing base (81afcb3) to head (2cfea63).
Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/local_object_storage/blobstor/fstree/head.go	53.03%	27 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3431      +/-   ##
==========================================
+ Coverage   21.04%   21.17%   +0.12%     
==========================================
  Files         707      707              
  Lines       52943    52979      +36     
==========================================
+ Hits        11143    11219      +76     
+ Misses      40983    40938      -45     
- Partials      817      822       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

pkg/local_object_storage/blobstor/fstree/fstree.go

pkg/local_object_storage/blobstor/fstree/head.go

pkg/local_object_storage/blobstor/fstree/fstree.go

pkg/local_object_storage/blobstor/fstree/head.go

Add a new `GetStream` method to the FSTree storage. This method allows for retrieving an object's payload as a stream (`io.ReadCloser`), which is significantly more memory-efficient for large objects as it avoids loading the entire payload into memory. Signed-off-by: Andrey Butusov <[email protected]>

Signed-off-by: Andrey Butusov <[email protected]>

Since #3383 and #3431, it is now not necessary to unmarshal the object to get its payload, and we can read the payload from reader to find the data. Closes #1724. Signed-off-by: Andrey Butusov <[email protected]>

Since #3383 and #3431, it is now not necessary to unmarshal the object to get its payload, and we can read the payload from reader to find the data. ``` goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor/fstree cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ FSTree_GetRange/size=10MB,off=1MB,len=4KB/regular-16 7457.9µ ± 8% 262.2µ ± 8% -96.48% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/compressed-16 7440.2µ ± 5% 261.5µ ± 10% -96.49% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/combined-16 9840.3µ ± 6% 355.8µ ± 4% -96.38% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/regular-16 8.242m ± 8% 3.977m ± 1% -51.75% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/compressed-16 7.814m ± 4% 3.979m ± 2% -49.07% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/combined-16 9.821m ± 4% 3.765m ± 3% -61.67% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/regular-16 10.432m ± 6% 3.987m ± 2% -61.78% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/compressed-16 10.721m ± 9% 3.982m ± 2% -62.85% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/combined-16 9.766m ± 5% 3.680m ± 6% -62.32% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/regular-16 77.48µ ± 2% 90.66µ ± 2% +17.00% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/compressed-16 77.42µ ± 5% 91.58µ ± 3% +18.29% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/combined-16 79.90µ ± 3% 90.45µ ± 1% +13.21% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/regular-16 80.28µ ± 2% 89.15µ ± 2% +11.05% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/compressed-16 79.36µ ± 3% 89.60µ ± 3% +12.91% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/combined-16 77.83µ ± 6% 91.00µ ± 1% +16.93% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/regular-16 79.47µ ± 2% 86.42µ ± 2% +8.75% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/compressed-16 79.30µ ± 4% 88.33µ ± 2% +11.40% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/combined-16 79.86µ ± 2% 90.45µ ± 1% +13.27% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/regular-16 10.294m ± 6% 1.856m ± 3% -81.97% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/compressed-16 10.059m ± 6% 1.848m ± 4% -81.63% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/combined-16 9.650m ± 6% 1.723m ± 6% -82.15% (p=0.000 n=10) geomean 1.199m 478.5µ -60.09% │ old.txt │ new.txt │ │ B/op │ B/op vs base │ FSTree_GetRange/size=10MB,off=1MB,len=4KB/regular-16 20495.52Ki ± 0% 43.91Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/compressed-16 20495.52Ki ± 0% 43.90Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/combined-16 20495.96Ki ± 0% 43.81Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/regular-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/compressed-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/combined-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/regular-16 20.02Mi ± 0% 10.04Mi ± 0% -49.85% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/compressed-16 20.02Mi ± 0% 10.04Mi ± 0% -49.85% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/combined-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/regular-16 17.62Ki ± 0% 48.31Ki ± 0% +174.24% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/compressed-16 17.62Ki ± 0% 48.31Ki ± 0% +174.24% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/combined-16 17.90Ki ± 0% 48.08Ki ± 0% +168.60% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/regular-16 17.77Ki ± 0% 47.95Ki ± 0% +169.80% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/compressed-16 17.77Ki ± 0% 47.95Ki ± 0% +169.80% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/combined-16 17.91Ki ± 0% 48.13Ki ± 0% +168.81% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/regular-16 17.92Ki ± 0% 44.77Ki ± 0% +149.83% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/compressed-16 17.92Ki ± 0% 44.77Ki ± 0% +149.83% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/combined-16 17.95Ki ± 1% 45.13Ki ± 0% +151.45% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/regular-16 20495.92Ki ± 0% 43.66Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/compressed-16 20495.92Ki ± 0% 43.66Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/combined-16 20495.94Ki ± 0% 43.75Ki ± 0% -99.79% (p=0.000 n=10) geomean 999.8Ki 214.7Ki -78.52% │ old.txt │ new.txt │ │ allocs/op │ allocs/op vs base │ FSTree_GetRange/size=10MB,off=1MB,len=4KB/regular-16 127.0 ± 0% 145.0 ± 0% +14.17% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/compressed-16 127.0 ± 0% 145.0 ± 0% +14.17% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/combined-16 139.0 ± 2% 141.0 ± 1% +1.44% (p=0.010 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/regular-16 149.0 ± 0% 148.0 ± 0% -0.67% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/compressed-16 149.0 ± 0% 148.0 ± 0% -0.67% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/combined-16 138.5 ± 3% 140.0 ± 2% +1.08% (p=0.027 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/regular-16 135.0 ± 0% 137.0 ± 0% +1.48% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/compressed-16 135.0 ± 0% 137.0 ± 0% +1.48% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/combined-16 137.0 ± 2% 141.0 ± 1% +2.92% (p=0.001 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/regular-16 131.0 ± 0% 145.0 ± 0% +10.69% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/compressed-16 131.0 ± 0% 145.0 ± 0% +10.69% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/combined-16 137.0 ± 1% 139.0 ± 1% +1.46% (p=0.007 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/regular-16 135.0 ± 0% 137.0 ± 0% +1.48% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/compressed-16 135.0 ± 0% 137.0 ± 0% +1.48% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/combined-16 137.5 ± 0% 140.5 ± 1% +2.18% (p=0.001 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/regular-16 139.0 ± 0% 129.0 ± 0% -7.19% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/compressed-16 139.0 ± 0% 129.0 ± 0% -7.19% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/combined-16 139.0 ± 2% 140.5 ± 2% ~ (p=0.058 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/regular-16 139.0 ± 0% 138.0 ± 0% -0.72% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/compressed-16 139.0 ± 0% 138.0 ± 0% -0.72% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=9MB,len=4KB/combined-16 138.0 ± 2% 140.5 ± 2% +1.81% (p=0.002 n=10) geomean 136.8 139.9 +2.26% ``` Closes #1724. Signed-off-by: Andrey Butusov <[email protected]>

Since #3383 and #3431, it is now not necessary to unmarshal the object to get its payload, and we can read the payload from reader to find the data. ``` goos: linux goarch: amd64 pkg: github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor/fstree cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ FSTree_GetRange/size=10MB,off=1MB,len=4KB/regular-16 8027.9µ ± 9% 264.8µ ± 5% -96.70% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/compressed-16 7608.3µ ± 5% 270.5µ ± 6% -96.45% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/combined-16 7391.8µ ± 6% 360.6µ ± 8% -95.12% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/regular-16 7.876m ± 4% 4.134m ± 3% -47.52% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/compressed-16 7.989m ± 6% 4.170m ± 3% -47.80% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/combined-16 7.530m ± 2% 4.038m ± 8% -46.37% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/regular-16 8.220m ± 6% 4.164m ± 4% -49.35% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/compressed-16 8.183m ± 6% 4.703m ± 6% -42.53% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/combined-16 7.508m ± 5% 4.325m ± 5% -42.40% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/regular-16 822.54µ ± 6% 83.04µ ± 4% -89.90% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/compressed-16 884.94µ ± 7% 92.61µ ± 5% -89.53% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/combined-16 1346.99µ ± 11% 84.62µ ± 10% -93.72% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/regular-16 908.2µ ± 15% 817.8µ ± 9% ~ (p=0.165 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/compressed-16 946.3µ ± 3% 806.2µ ± 4% -14.80% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/combined-16 1290.4µ ± 9% 914.6µ ± 11% -29.12% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/regular-16 955.0µ ± 3% 863.4µ ± 4% -9.60% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/compressed-16 948.7µ ± 4% 869.5µ ± 5% -8.35% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/combined-16 1318.8µ ± 13% 955.9µ ± 7% -27.52% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/regular-16 79.55µ ± 5% 89.11µ ± 2% +12.01% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/compressed-16 78.48µ ± 5% 87.90µ ± 2% +12.01% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/combined-16 79.15µ ± 4% 92.59µ ± 3% +16.98% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/regular-16 81.78µ ± 4% 92.22µ ± 4% +12.77% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/compressed-16 79.80µ ± 11% 90.99µ ± 3% +14.02% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/combined-16 92.39µ ± 9% 92.73µ ± 2% ~ (p=0.280 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/regular-16 92.11µ ± 1% 89.30µ ± 2% -3.06% (p=0.002 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/compressed-16 80.11µ ± 17% 91.34µ ± 2% ~ (p=0.105 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/combined-16 79.40µ ± 5% 90.61µ ± 4% +14.12% (p=0.000 n=10) geomean 871.6µ 399.9µ -54.12% │ old.txt │ new.txt │ │ B/op │ B/op vs base │ FSTree_GetRange/size=10MB,off=1MB,len=4KB/regular-16 20495.77Ki ± 0% 43.65Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/compressed-16 20495.77Ki ± 0% 43.65Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/combined-16 20496.00Ki ± 0% 43.81Ki ± 0% -99.79% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/regular-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/compressed-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/combined-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/regular-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/compressed-16 20.02Mi ± 0% 10.04Mi ± 0% -49.84% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/combined-16 20.02Mi ± 0% 10.04Mi ± 0% -49.85% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/regular-16 2063.83Ki ± 0% 43.65Ki ± 0% -97.89% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/compressed-16 2063.83Ki ± 0% 43.65Ki ± 0% -97.88% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/combined-16 2063.98Ki ± 0% 43.89Ki ± 0% -97.87% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/regular-16 2.015Mi ± 0% 1.039Mi ± 0% -48.45% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/compressed-16 2.015Mi ± 0% 1.039Mi ± 0% -48.45% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/combined-16 2.016Mi ± 0% 1.039Mi ± 0% -48.46% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/regular-16 2.015Mi ± 0% 1.039Mi ± 0% -48.44% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/compressed-16 2.015Mi ± 0% 1.039Mi ± 0% -48.44% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/combined-16 2.016Mi ± 0% 1.039Mi ± 0% -48.46% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/regular-16 17.97Ki ± 0% 47.68Ki ± 0% +165.35% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/compressed-16 17.97Ki ± 0% 47.68Ki ± 0% +165.35% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/combined-16 17.91Ki ± 0% 48.13Ki ± 0% +168.71% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/regular-16 18.12Ki ± 0% 48.09Ki ± 0% +165.42% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/compressed-16 18.12Ki ± 0% 48.09Ki ± 0% +165.42% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/combined-16 17.90Ki ± 1% 48.12Ki ± 0% +168.78% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/regular-16 18.15Ki ± 0% 45.07Ki ± 0% +148.34% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/compressed-16 18.15Ki ± 0% 45.07Ki ± 0% +148.34% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/combined-16 17.92Ki ± 1% 45.11Ki ± 0% +151.67% (p=0.000 n=10) geomean 913.5Ki 306.3Ki -66.47% │ old.txt │ new.txt │ │ allocs/op │ allocs/op vs base │ FSTree_GetRange/size=10MB,off=1MB,len=4KB/regular-16 135.0 ± 0% 138.0 ± 0% +2.22% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/compressed-16 135.0 ± 0% 138.0 ± 0% +2.22% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=1MB,len=4KB/combined-16 140.0 ± 1% 140.5 ± 1% +0.36% (p=0.038 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/regular-16 142.0 ± 0% 145.0 ± 0% +2.11% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/compressed-16 142.0 ± 0% 145.0 ± 0% +2.11% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=10MB/combined-16 138.0 ± 1% 140.5 ± 2% ~ (p=0.063 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/regular-16 139.0 ± 0% 148.0 ± 0% +6.47% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/compressed-16 139.0 ± 0% 148.0 ± 0% +6.47% (p=0.000 n=10) FSTree_GetRange/size=10MB,off=Empty,len=Empty/combined-16 139.0 ± 2% 139.0 ± 1% ~ (p=0.773 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/regular-16 135.0 ± 0% 134.0 ± 0% -0.74% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/compressed-16 135.0 ± 0% 134.0 ± 0% -0.74% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=1KB,len=4KB/combined-16 139.5 ± 2% 140.5 ± 2% +0.72% (p=0.030 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/regular-16 127.0 ± 0% 141.0 ± 0% +11.02% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/compressed-16 127.0 ± 0% 141.0 ± 0% +11.02% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=1MB/combined-16 138.0 ± 2% 139.5 ± 1% ~ (p=0.195 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/regular-16 127.0 ± 0% 144.0 ± 0% +13.39% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/compressed-16 127.0 ± 0% 144.0 ± 0% +13.39% (p=0.000 n=10) FSTree_GetRange/size=1MB,off=Empty,len=Empty/combined-16 139.0 ± 1% 140.0 ± 2% ~ (p=0.146 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/regular-16 139.0 ± 0% 129.0 ± 0% -7.19% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/compressed-16 139.0 ± 0% 129.0 ± 0% -7.19% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=4KB/combined-16 138.0 ± 1% 140.5 ± 1% +1.81% (p=0.001 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/regular-16 142.0 ± 0% 140.0 ± 0% -1.41% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/compressed-16 142.0 ± 0% 140.0 ± 0% -1.41% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=Empty,len=Empty/combined-16 137.5 ± 3% 140.0 ± 2% ~ (p=0.091 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/regular-16 143.0 ± 0% 140.0 ± 0% -2.10% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/compressed-16 143.0 ± 0% 140.0 ± 0% -2.10% (p=0.000 n=10) FSTree_GetRange/size=4KB,off=1KB,len=1KB/combined-16 138.0 ± 1% 140.5 ± 2% +1.81% (p=0.030 n=10) geomean 137.1 139.9 +2.01% ``` Closes #1724. Signed-off-by: Andrey Butusov <[email protected]>

Network API always had a concept of payload stream, FSTree can also provide it now after #3431. So implement `GetStream` throughout the storage stack (FSTree - Shard - Engine - Service) to enable memory-efficient object retrieval without loading large payloads into memory. Closes #3439. Signed-off-by: Andrey Butusov <[email protected]>

Network API always had a concept of payload stream, FSTree can also provide it now after #3431. So use `GetStream` in object service to enable memory-efficient object retrieval without loading large payloads into memory. Closes #3439. Signed-off-by: Andrey Butusov <[email protected]>

End-rey self-assigned this Jun 27, 2025

End-rey requested review from carpawell, cthulhu-rider and roman-khimov as code owners June 27, 2025 21:31

roman-khimov reviewed Jun 30, 2025

View reviewed changes

pkg/local_object_storage/blobstor/fstree/fstree.go Outdated Show resolved Hide resolved

End-rey force-pushed the fstree-stream-api branch from 66d69b5 to 70f7914 Compare July 1, 2025 08:32

End-rey requested a review from roman-khimov July 1, 2025 08:36

roman-khimov approved these changes Jul 1, 2025

View reviewed changes

pkg/local_object_storage/blobstor/fstree/head.go Show resolved Hide resolved

pkg/local_object_storage/blobstor/fstree/head.go Outdated Show resolved Hide resolved

cthulhu-rider reviewed Jul 1, 2025

View reviewed changes

End-rey force-pushed the fstree-stream-api branch 2 times, most recently from 94ce490 to 2cfea63 Compare July 1, 2025 12:17

End-rey added 2 commits July 1, 2025 15:17

cli: remove trailing spaces from descriptions

2cfea63

Signed-off-by: Andrey Butusov <[email protected]>

End-rey requested a review from cthulhu-rider July 1, 2025 12:24

cthulhu-rider approved these changes Jul 1, 2025

View reviewed changes

carpawell approved these changes Jul 1, 2025

View reviewed changes

roman-khimov merged commit 3c09860 into master Jul 1, 2025
23 checks passed

roman-khimov deleted the fstree-stream-api branch July 1, 2025 12:43

roman-khimov mentioned this pull request Jul 2, 2025

Stream GET replies #3439

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fstree: serve `Stream` from the storage #3431

fstree: serve `Stream` from the storage #3431

Uh oh!

End-rey commented Jun 27, 2025

Uh oh!

codecov bot commented Jun 27, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fstree: serve Stream from the storage #3431

fstree: serve Stream from the storage #3431

Uh oh!

Conversation

End-rey commented Jun 27, 2025

Uh oh!

codecov bot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fstree: serve `Stream` from the storage #3431

fstree: serve `Stream` from the storage #3431

codecov bot commented Jun 27, 2025 •

edited

Loading